TY - JOUR

T1 - Evaluation of the simulation based training quality assurance tool (SBT-QA10) as a measure of learners’ perceptions during the action phase of simulation

AU - Ekelund, Kim

AU - O’Regan, Stephanie

AU - Dieckmann, Peter

AU - Østergaard, Doris

AU - Watterson, Leonie

N1 - Funding Information: Dieckmann holds a professorship with the University of Stavanger in Norway. This position is paid for by an unconditional grant from the Laerdal foundation to the University. Dieckmann leads the EuSim group, a network of simulation enthusiasts and centres to provide faculty development programmes on an international basis. The remaining authors have no competing interests Publisher Copyright: © 2023, The Author(s).

PY - 2023/12

Y1 - 2023/12

N2 - Background: In an earlier interview-based study the authors identified that learners experience one or more of eight explicit perceptual responses during the active phase of simulation-based training (SBT) comprising a sense: of belonging to instructor and group, of being under surveillance, of having autonomy and responsibility for patient management, of realism, of an understanding of the scenario in context, of conscious mental effort, of control of attention, and of engagement with task. These were adapted into a ten-item questionnaire: the Simulation Based Training Quality Assurance Tool (SBT-QA10) to allow monitoring of modifiable factors that may impact upon learners’ experiences. This study assessed the construct validity evidence of the interpretation of the results when using SBT-QAT10. Materials and methods: Recently graduated doctors and nurses participating in a SBT course on the topic of the deteriorating patient completed the SBT-QAT10 immediately following their participation in the scenarios. The primary outcome measure was internal consistency of the questionnaire items and their correlation to learners’ satisfaction scores. A secondary outcome measure compared the impact of allocation to active versus observer role. Results: A total of 349 questionnaires were returned by 96 course learners. The median of the total score for the ten perception items (TPS) was 39 (out of 50), with no significant difference between the scenarios. We identified fair and positive correlations between nine of the 10 items and the SBT-QA10-TPS, the exception being “mental effort”. Compared to observers, active learners reported significantly more positive perceptions related to belonging to the team and interaction with the instructor, their sense of acting independently, and being focused. The questionnaire items were poorly correlated with the two measures of global satisfaction. Conclusion: Except for the item for mental effort, the QA10-TPS measures learners’ experiences during the active phase of simulation scenarios that are associated with a positive learning experience. The tool may have utility to learners, instructors, and course providers by informing subsequent debriefing and reflection upon practice for learners and faculty. The relationship between these perceptions and commonly used measures of satisfaction remains poorly understood raising questions about the value of the latter.

AB - Background: In an earlier interview-based study the authors identified that learners experience one or more of eight explicit perceptual responses during the active phase of simulation-based training (SBT) comprising a sense: of belonging to instructor and group, of being under surveillance, of having autonomy and responsibility for patient management, of realism, of an understanding of the scenario in context, of conscious mental effort, of control of attention, and of engagement with task. These were adapted into a ten-item questionnaire: the Simulation Based Training Quality Assurance Tool (SBT-QA10) to allow monitoring of modifiable factors that may impact upon learners’ experiences. This study assessed the construct validity evidence of the interpretation of the results when using SBT-QAT10. Materials and methods: Recently graduated doctors and nurses participating in a SBT course on the topic of the deteriorating patient completed the SBT-QAT10 immediately following their participation in the scenarios. The primary outcome measure was internal consistency of the questionnaire items and their correlation to learners’ satisfaction scores. A secondary outcome measure compared the impact of allocation to active versus observer role. Results: A total of 349 questionnaires were returned by 96 course learners. The median of the total score for the ten perception items (TPS) was 39 (out of 50), with no significant difference between the scenarios. We identified fair and positive correlations between nine of the 10 items and the SBT-QA10-TPS, the exception being “mental effort”. Compared to observers, active learners reported significantly more positive perceptions related to belonging to the team and interaction with the instructor, their sense of acting independently, and being focused. The questionnaire items were poorly correlated with the two measures of global satisfaction. Conclusion: Except for the item for mental effort, the QA10-TPS measures learners’ experiences during the active phase of simulation scenarios that are associated with a positive learning experience. The tool may have utility to learners, instructors, and course providers by informing subsequent debriefing and reflection upon practice for learners and faculty. The relationship between these perceptions and commonly used measures of satisfaction remains poorly understood raising questions about the value of the latter.

KW - Construct validity

KW - Learners’ perception

KW - Observer role

KW - Quality Assurance Tool

KW - Questionnaire development

KW - Simulation-based learning

KW - Simulation-based training

U2 - 10.1186/s12909-023-04273-6

DO - 10.1186/s12909-023-04273-6

M3 - Journal article

C2 - 37127593

AN - SCOPUS:85158043770

VL - 23

JO - BMC Medical Education

JF - BMC Medical Education

SN - 1472-6920

IS - 1

M1 - 290

ER -