Introduction: Cardiovascular diseases are globally a major cause of death. Magnesium deficiency is associated with several diseases including cardiovascular diseases. Objective: To examine if a low concentration of magnesium in drinking water is associated with increased cardiovascular mortality and mortality due to acute myocardial infarction and stroke. Methods: A nationwide population-based cohort study using national health registries was used. A total of 4,274,132 individuals aged 30 years or more were included. Magnesium concentration in drinking water was estimated by linkage of residential addresses in the period 2005–2016 with the national drinking water quality monitoring database. The association between magnesium concentration in drinking water and cardiovascular mortality and mortality due to acute myocardial infarction and stroke was examined using a Poisson regression of number of deaths and logarithmic transformation of follow-up time as offset. The incidence rate ratio (IRR) was adjusted for differences in age, sex, calendar year, cohabitation, country of origin, and socioeconomic status. Results: Median magnesium concentration in drinking water at inclusion was 12.4 mg/L (range: 1.37–54.2 mg/L). The adjusted IRR for cardiovascular mortality was 0.96 (95% CI: 0.94; 0.97) for the lowest magnesium quintile (<6.5 mg/L) as compared to the highest magnesium quintile (>21.9 mg/L). The adjusted IRR for mortality due to acute myocardial infarction and stroke was 1.22 (1.17; 1.27) and 0.96 (0.93; 0.99), respectively, for the lowest magnesium quintile as compared to the highest quintile A decreasing mortality due to acute myocardial infarction was seen with an increasing magnesium concentration in a dose–response manner. Conclusion: Low concentrations of magnesium in drinking water were associated with an increased mortality due to acute myocardial infarction. Low concentrations of magnesium in drinking water were associated with decreased cardiovascular mortality, and mortality due to stroke.