Germ cell neoplasms, a cancer type most common in young individuals, are found mainly in the gonads. Testicular tumors in adolescents and young adults are derived from a preinvasive precursor cell called carcinoma in situ (CIS) or intratubular germ cell neoplasia, unclassified (ITGCNU). CIS has features of immature fetal germ cells, and is believed to originate from developmentally arrested primordial germ cells or gonocytes. In most cases, the arrest is caused by gonadal dysgenesis, a multifactorial and complex syndrome that has a broad spectrum of phenotypes, ranging from moderate impairment of spermatogenesis to a variety of intersex disorders. Testicular dysgenesis syndrome may be a result of inherited genetic aberrations, but in most cases the pathogenesis is unknown, although it is suspected that genes involved in sex differentiation go through transcriptional or functional deregulation caused by external environmental factors. The effects of these factors are likely modulated by genomic variations (polymorphisms), thus explaining the individual- and population-level differences. In this article, the existing evidence concerning the role of the sex chromosomes is reviewed, with emphasis on aneuploidy and structural aberrations, which carry a high risk of germ cell neoplasia. Less is known about the role of single genes or gene polymorphisms in germ cell development and neoplastic transformation. We hypothesize that factors disturbing gonadal development (including imbalance between the Y and X chromosomes) may lead to impaired germ cell differentiation and prolonged expression of pluripotency genes, and may increase the risk of neoplastic transformation of germ cells.