Context: Controversy exists regarding associations between early-life growth patterns and timing of puberty. Objective: This work aims to investigate associations between birth anthropometry, early growth patterns, and onset/progression of pubertal milestones in boys and girls. Methods: Among children examined at birth (1997-2003) and at age 36 months in a mother-child cohort, pubertal Tanner stages (B1-5, PH1-5, G1-5) and testicular volume were examined by trained physicians at 1 to 5 follow-up examinations during childhood and adolescence (672 girls and 846 boys, 2006-2013). With parametric survival models we analyzed associations between birth weight, changes in SD scores (SDS) from birth to 36 months (ΔSDS 0-36 > 0.67 SD defining catch-up growth), and age at pubertal onset/attainment of late pubertal stages/menarche. Results: A 1-kg higher birth weight was associated with earlier onset of B2+ (thelarche): -3.9 months (CI, -6.7 to -1.1 months), G2+ (gonadarche): -2.7 months (-5.3 to -0.1 months), Tvol3+ (testis size > 3 mL): -2.8 months (CI, -4.9 to -0.7 months), but with later G4+ and PH4+ in boys, and a slower progression from B2 to menarche (5.3 months [CI, 1.2 to 9.4 months]) in girls. Catch-up growth was associated with earlier PH2+ (pubarche) in girls (-4.1 months [CI, -7.6 to -0.6 months]), earlier PH2+ in boys (-3.4 months [CI, -6.6 to -0.2 months]), faster progression from B2 to menarche in girls (-9.1 months [CI, 14.6 to 3.5 months]), and earlier G4+ and PH4+ in boys. Conclusion: Associations between birthweight and infancy catch-up growth differed for gonadarche and pubarche, and for early and late pubertal markers, with similar patterns in both sexes.