Collecting Validity Evidence for the Assessment of Mastery Learning in Simulation-Based Ultrasound Training
Publikation: Bidrag til tidsskrift › Tidsskriftartikel › Forskning › fagfællebedømt
Standard
Collecting Validity Evidence for the Assessment of Mastery Learning in Simulation-Based Ultrasound Training. / Dyre, L; Nørgaard, L N; Tabor, A; Madsen, M E; Sørensen, J L; Ringsted, C; Tolsgaard, M.
I: Ultraschall in der Medizin, Bind 37, Nr. 4, 2016, s. 386-392.Publikation: Bidrag til tidsskrift › Tidsskriftartikel › Forskning › fagfællebedømt
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - Collecting Validity Evidence for the Assessment of Mastery Learning in Simulation-Based Ultrasound Training
AU - Dyre, L
AU - Nørgaard, L N
AU - Tabor, A
AU - Madsen, M E
AU - Sørensen, J L
AU - Ringsted, C
AU - Tolsgaard, M
N1 - © Georg Thieme Verlag KG Stuttgart · New York.
PY - 2016
Y1 - 2016
N2 - Purpose: To collect validity evidence for the assessment of mastery learning on a virtual reality transabdominal ultrasound simulator. Materials and Methods: We assessed the validity evidence using Messick's framework for validity. The study included 20 novices and 9 ultrasound experts who all completed 10 obstetric training modules on a transabdominal ultrasound simulator that provided automated measures of performance for each completed module (i. e., simulator metrics). Differences in the performance of the two groups were used to identify simulator metrics with validity evidence for the assessment of mastery learning. The novices continued to practice until they had attained mastery learning level. Results: One-third of the simulator metrics discriminated between the two groups. The median simulator scores from a maximum of 40 metrics were 17.5 percent (range 0 - 45.0 percent) for novices and 90.0 percent (range 85.0 - 97.5) for experts, p < 0.001. Internal consistency was high, with a Cronbach's alpha value of 0.98. The test/retest reliability gave an intra-class correlation coefficient (ICC) of 0.62 for novices who reached the mastery learning level twice. Novices reached the mastery learning level within a median of 4 attempts (range 3 - 8) corresponding to a median of 252 minutes of simulator training (range 211 - 394 minutes). Conclusion: This study found that validity evidence for the assessment of mastery learning in simulation-based ultrasound training can be demonstrated and that ultrasound novices can attain mastery learning levels with less than 5 hours of training. Only one-third of the standard simulator metrics discriminated between different levels of competence.
AB - Purpose: To collect validity evidence for the assessment of mastery learning on a virtual reality transabdominal ultrasound simulator. Materials and Methods: We assessed the validity evidence using Messick's framework for validity. The study included 20 novices and 9 ultrasound experts who all completed 10 obstetric training modules on a transabdominal ultrasound simulator that provided automated measures of performance for each completed module (i. e., simulator metrics). Differences in the performance of the two groups were used to identify simulator metrics with validity evidence for the assessment of mastery learning. The novices continued to practice until they had attained mastery learning level. Results: One-third of the simulator metrics discriminated between the two groups. The median simulator scores from a maximum of 40 metrics were 17.5 percent (range 0 - 45.0 percent) for novices and 90.0 percent (range 85.0 - 97.5) for experts, p < 0.001. Internal consistency was high, with a Cronbach's alpha value of 0.98. The test/retest reliability gave an intra-class correlation coefficient (ICC) of 0.62 for novices who reached the mastery learning level twice. Novices reached the mastery learning level within a median of 4 attempts (range 3 - 8) corresponding to a median of 252 minutes of simulator training (range 211 - 394 minutes). Conclusion: This study found that validity evidence for the assessment of mastery learning in simulation-based ultrasound training can be demonstrated and that ultrasound novices can attain mastery learning levels with less than 5 hours of training. Only one-third of the standard simulator metrics discriminated between different levels of competence.
U2 - 10.1055/s-0041-107976
DO - 10.1055/s-0041-107976
M3 - Journal article
C2 - 27112623
VL - 37
SP - 386
EP - 392
JO - Ultraschall in der Medizin
JF - Ultraschall in der Medizin
SN - 0172-4614
IS - 4
ER -
ID: 161804831