TY - JOUR

T1 - Development and validation of a theoretical test of proficiency for video-assisted thoracoscopic surgery (VATS) lobectomy

AU - Savran, Mona M

AU - Hansen, Henrik Jessen

AU - Horsleben Petersen, René

AU - Walker, William

AU - Schmid, Thomas

AU - Bojsen, Signe Rolskov

AU - Konge, Lars

PY - 2015/9

Y1 - 2015/9

N2 - BACKGROUND: Testing stimulates learning, improves long-term retention, and promotes technical performance. No purpose-orientated test of competence in the theoretical aspects of VATS lobectomy has previously been presented. The purpose of this study was, therefore, to develop and gather validity evidence for a theoretical test on VATS lobectomy consisting of multiple-choice questions.METHODS: Four European VATS lobectomy experts were interviewed to explore their views on important theoretical VATS lobectomy knowledge (step 1). This information was used to construct the test items in compliance with existing guidelines for multiple-choice questions (step 2). The experts rated the relevance of the items to confirm content validity in a modified Delphi approach (step 3). Finally, the test was administered to physicians, who were categorised into different experience levels based on their experience in VATS procedures overall and in VATS lobectomies specifically. Their answers were used to achieve construct validity (step 4).RESULTS: Initially, 81 items were constructed and two Delphi iterations reduced the test to 50 items. Item analysis led to the exclusion of 19 items and the mean discrimination index of the 31 final items was 0.26. Cronbach's alpha for internal consistency was 0.75. The mean item difficulty was calculated to 0.63. According to performed VATS procedures, significantly different test performances were detected when comparing the group performances (p = 0.002) and the experts performed significantly better than the novices (p < 0.001) and intermediates (p = 0.01). In the category of performed VATS lobectomies, significant group performances were also found. In this category, the experts were also significantly better than the novices (p < 0.001), the trainees (p = 0.002), and the intermediates (p = 0.01).CONCLUSIONS: This study led to the development of a theoretical test on VATS lobectomy consisting of multiple-choice questions. Both content and construct validity evidence were established.

AB - BACKGROUND: Testing stimulates learning, improves long-term retention, and promotes technical performance. No purpose-orientated test of competence in the theoretical aspects of VATS lobectomy has previously been presented. The purpose of this study was, therefore, to develop and gather validity evidence for a theoretical test on VATS lobectomy consisting of multiple-choice questions.METHODS: Four European VATS lobectomy experts were interviewed to explore their views on important theoretical VATS lobectomy knowledge (step 1). This information was used to construct the test items in compliance with existing guidelines for multiple-choice questions (step 2). The experts rated the relevance of the items to confirm content validity in a modified Delphi approach (step 3). Finally, the test was administered to physicians, who were categorised into different experience levels based on their experience in VATS procedures overall and in VATS lobectomies specifically. Their answers were used to achieve construct validity (step 4).RESULTS: Initially, 81 items were constructed and two Delphi iterations reduced the test to 50 items. Item analysis led to the exclusion of 19 items and the mean discrimination index of the 31 final items was 0.26. Cronbach's alpha for internal consistency was 0.75. The mean item difficulty was calculated to 0.63. According to performed VATS procedures, significantly different test performances were detected when comparing the group performances (p = 0.002) and the experts performed significantly better than the novices (p < 0.001) and intermediates (p = 0.01). In the category of performed VATS lobectomies, significant group performances were also found. In this category, the experts were also significantly better than the novices (p < 0.001), the trainees (p = 0.002), and the intermediates (p = 0.01).CONCLUSIONS: This study led to the development of a theoretical test on VATS lobectomy consisting of multiple-choice questions. Both content and construct validity evidence were established.

U2 - 10.1007/s00464-014-3975-y

DO - 10.1007/s00464-014-3975-y

M3 - Journal article

C2 - 25427417

VL - 29

SP - 2598

EP - 2604

JO - Surgical Endoscopy and Other Interventional Techniques

JF - Surgical Endoscopy and Other Interventional Techniques

SN - 0930-2794

IS - 9

ER -