TY - JOUR

T1 - Solid validity evidence for two tools assessing competences in musculoskeletal ultrasound

T2 - a validity study

AU - Carstensen, Stine Maya Dreier

AU - Just, Søren Andreas

AU - Pfeiffer-Jensen, Mogens

AU - Østergaard, Mikkel

AU - Konge, Lars

AU - Terslev, Lene

N1 - Publisher Copyright: © The Author(s) 2023.

PY - 2024

Y1 - 2024

N2 - Objectives: Musculoskeletal ultrasound (MSUS) is increasingly used by rheumatologists in daily clinical practice. However, MSUS is only valuable in trained hands, and assessment of trainee competences is therefore essential before independent practice. Thus, this study aimed to establish validity evidence for the EULAR and the Objective Structured Assessment of Ultrasound Skills (OSAUS) tools used for assessing MSUS competences. Methods: Thirty physicians with different levels of MSUS experience (novices, intermediates, and experienced) performed four MSUS examinations of different joint areas on the same rheumatoid arthritis patient. All examinations were video recorded (n=120), anonymized, and subsequently assessed in random order by two blinded raters using first the OSAUS assessment tool followed by the EULAR tool 1 month after. Results: The inter-rater reliability between the two raters was high for both the OSAUS and EULAR tools, with a Pearson correlation coefficient (PCC) of 0.807 and 0.848, respectively. Both tools demonstrated excellent inter-case reliability, with a Cronbach's alpha of 0.970 for OSAUS and 0.964 for EULAR. Furthermore, there was a strong linear correlation between the OSAUS and the EULAR performance scores and the participants' experience levels (R2=0.897 and R2=0.868, respectively) and a significant discrimination between different MSUS experience levels (P<0.001 for both). Conclusions: MSUS operator competences can be assessed reliably and valid using either the OSAUS or the EULAR assessment tool, thereby allowing a uniform competency-based MSUS education in the future. Although both tools demonstrated high inter-rater reliability, the EULAR tool was superior to OSAUS.

AB - Objectives: Musculoskeletal ultrasound (MSUS) is increasingly used by rheumatologists in daily clinical practice. However, MSUS is only valuable in trained hands, and assessment of trainee competences is therefore essential before independent practice. Thus, this study aimed to establish validity evidence for the EULAR and the Objective Structured Assessment of Ultrasound Skills (OSAUS) tools used for assessing MSUS competences. Methods: Thirty physicians with different levels of MSUS experience (novices, intermediates, and experienced) performed four MSUS examinations of different joint areas on the same rheumatoid arthritis patient. All examinations were video recorded (n=120), anonymized, and subsequently assessed in random order by two blinded raters using first the OSAUS assessment tool followed by the EULAR tool 1 month after. Results: The inter-rater reliability between the two raters was high for both the OSAUS and EULAR tools, with a Pearson correlation coefficient (PCC) of 0.807 and 0.848, respectively. Both tools demonstrated excellent inter-case reliability, with a Cronbach's alpha of 0.970 for OSAUS and 0.964 for EULAR. Furthermore, there was a strong linear correlation between the OSAUS and the EULAR performance scores and the participants' experience levels (R2=0.897 and R2=0.868, respectively) and a significant discrimination between different MSUS experience levels (P<0.001 for both). Conclusions: MSUS operator competences can be assessed reliably and valid using either the OSAUS or the EULAR assessment tool, thereby allowing a uniform competency-based MSUS education in the future. Although both tools demonstrated high inter-rater reliability, the EULAR tool was superior to OSAUS.

KW - assessment

KW - competency-based education

KW - education research

KW - medical education

KW - musculoskeletal ultrasound

KW - training

KW - ultrasound competence

U2 - 10.1093/rheumatology/kead286

DO - 10.1093/rheumatology/kead286

M3 - Journal article

C2 - 37307078

AN - SCOPUS:85186362989

VL - 63

SP - 765

EP - 771

JO - Rheumatology

JF - Rheumatology

SN - 1462-0324

IS - 3

ER -