PURPOSE:  The purpose of this study was two-fold: (1) To determine how the number of ultrasound scans a trainee has completed predicts the trainee's diagnostic accuracy when performing transvaginal ultrasound examinations, and (2) to examine the utility of simulation-based assessment of ultrasound competence for determining readiness for independently performing examinations.

MATERIALS AND METHODS:  101 OB/GYN trainees were surveyed regarding their clinical experience and the number of scans they had completed. All participants completed five different cases on a transvaginal virtual-reality ultrasound simulator (Scantrainer, Medaphor). The participants' diagnostic accuracy was recorded and expert raters evaluated their performance using the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale. The utility of simulation-based assessments was assessed with respect to reliability, validity, acceptability, and costs. The main outcome was diagnostic accuracy for five different ultrasound cases.

RESULTS:  Although the number of scans was associated with diagnostic accuracy (p = 0.006), it was a poor predictor (AUC 0.69) of diagnostic accuracy. Only 56.6 % (n = 34) of participants who had more than 100 transvaginal scans demonstrated a diagnostic accuracy of 0.80 or above. The reliability of the OSAUS assessments was high (ICC 0.82) and the majority of participants supported the use of simulation-based assessments for future licensing exams (70.3 %). The running costs of simulation-based assessments (154 EUR per participant) were lower than for practical examinations using real patients.

CONCLUSION:  The number of completed ultrasound scans was a poor predictor of the trainees' diagnostic accuracy. Instead, simulation-based assessments can be used to ensure that trainees are ready for independently performing future scans.