Patients with non-ischemic systolic heart failure (HF) have increased risk of sudden cardiovascular death (SCD). The initiation and substrate for ventricular arrhythmias remains poorly understood. Our purpose was to describe the relationship between cardiac magnetic resonance (CMR) late gadolinium enhancement (LGE) and Holter recorded ventricular arrhythmic activity. Patients from the DANISH trial underwent a Holter-recording and a CMR-scan. The presence of non-sustained ventricular tachycardia (NSVT) and premature ventricular contractions (PVC) were examined in relation to presence and amount of LGE. Outcome measures were all-cause mortality and SCD. Overall, 180 patients were included. LGE was present in 86 (47%). NSVT occurred in 72 (40%), not different according to LGE status (p = 0.65). The amount of LGE was not correlated to the occurrence of NSVT (p = 0.40). The occurrence of couplet PVCs (p = 0.997), frequent PVCs (p = 0.12), PVCs in bigemini (p = 0.29), in trigemini (p = 0.26), or in quadrimini (p = 0.35) did not differ according to LGE status. LGE was significantly associated with risk of all-cause mortality (HR 2.14; 95% CI 1.05–4.37, p = 0.04). NSVT did not increase risk of all-cause mortality in either patients with LGE (HR 1.00; 95% CI 0.46–2.16, p = 0.996) or without LGE (HR 1.37; 95% CI 0.46–4.08, p = 0.57). There was no interaction between LGE and NSVT for the risk of all-cause mortality (p = 0.62). In patients with non-ischemic systolic HF there was no relationship between the presence of LGE and NSVT or any other Holter recorded ventricular tachyarrhythmia. LGE was associated with increased risk of mortality, independent of the presence of NSVT.