Background: The number of women with congenital heart disease (CHD) becoming pregnant are increasing. Although menstrual irregularities appear to occur more often in these patients, knowledge on their fertility is limited. In this nationwide cohort study, we evaluated the risk of impaired fertility in women with CHD compared with unaffected women using time to pregnancy (TTP). Methods: The Danish National Birth Cohort (DNBC) of pregnant women constituted the study population. Information on TTP and use of medically assisted reproduction (MAR) treatment was reported at a first trimester interview. Women with CHD were identified by linkage to the Danish National Patient Registry. TTP was divided into three categories; 0–5 months, 6–12 months (i.e. subfertile), and > 12 months or use of MAR treatment (i.e. infertile). Relative risk ratios (RRR) for subfertility and infertility with 95% confidence intervals were estimated using multinomial logistic regression. Results: Among 93,832 pregnancies in 84,922 women, CHD was diagnosed in 333 women (0.4%), contributing with 360 pregnancies. The CHD was of simple complexity in 291 women (87.4%). No association was found between CHD and longer TTP (RRR of 1.02 (95% CI: 0.75–1.40) for subfertility, and RRR of 0.86 (95% CI: 0.61–1.20) for infertility). Similar was observed when comparing women with simple CHD and unaffected women. The number of women with complex CHD was too low for evaluation. Conclusions: Women with CHD had no increased risk of impaired fertility, assessed by TTP, when compared with unaffected women. Separate analysis of women with complex CHD was hampered by low numbers.