Magnetic resonance imaging (CMR) is applied in mitral valve regurgitation (MR) to quantify regurgitation volume/fraction and cardiac volumes, but individual scallop pathology is evaluated by echocardiography. To evaluate CMR for determination of individual scallop pathology, interrater variability on evaluation of scallop pathology from echocardiography and a standard clinical CMR protocol including a transversal stack was compared. 318 mitral scallops from 53 patients with primary MR were evaluated by two cardiologists evaluating echocardiography scans and two other cardiologists evaluating CMR scans (blinded). Inter-rater variability was determined with percentage agreement and Cohen’s kappa. In evaluable scallops, interrater agreement on the diagnosis of a prolapsing and/or flail scallop was 77–87% and kappa values of 0.27–0.67, irrespective of physician or modality. Important differences between modalities were primarily related to CMR-evaluators judging the A3 and the P3 to be normal when echocardiography demonstrated prolapsing or even flail scallops; poor imaging of calcification; and flailed scallops occasionally being undetected with CMR since the flow-voids may mask the scallop. Inter-rater agreement for scallop pathology in primary MR is comparable for echocardiography and standard magnetic resonance imaging scans, but CMR has important pitfalls relating to evaluation of A3 and P3 scallops, and suffers from poor visualization of calcification and lower spatial resolution than echo. CMR with standard planes cannot replace CMR with longitudinal planes or echo for the evaluation of specific scallop pathology in severe primary MR.