A total of 1,215 young Danish men aged 18-28 years were recruited between 2008 and 2012 when they attended a compulsory medical examination to determine their fitness for military service. The participants delivered a semen sample, had a blood sample drawn, and underwent a physical examination. They responded to questionnaires including information on marijuana and recreational drug use during the past 3 months (no use, use once per week or less, or use more than once per week). A total of 45% had smoked marijuana within the last 3 months. Regular marijuana smoking more than once per week was associated with a 28% (95% confidence interval (CI): -48, -1) lower sperm concentration and a 29% (95% CI: -46, -1) lower total sperm count after adjustment for confounders. The combined use of marijuana more than once per week and other recreational drugs reduced the sperm concentration by 52% (95% CI: -68, -27) and total sperm count by 55% (95% CI: -71, -31). Marijuana smokers had higher levels of testosterone within the same range as cigarette smokers. Our findings are of public interest as marijuana use is common and may be contributing to recent reports of poor semen quality.