OBJECTIVE: A gold standard of skills required for nurse-administered propofol sedation (NAPS) for gastroenterological endoscopic procedures has been proposed but not established. Due to the potentially hazardous nature of NAPS, an assessment tool is needed to objectively judge the adequacy of training and for future certification. The aim of this study was to develop an assessment tool for measuring competency in propofol sedation and to explore the reliability and validity of the tool.

MATERIAL AND METHODS: The nurse-administered propofol assessment tool (NAPSAT) was developed in a Delphi-like fashion. Consensus was achieved on 17 items. Validity evidence was gathered in a case-control study in a full-scale simulation setting. Six experienced nurses and six novice nurses were filmed in two scenarios for assessment according to the assessment tool by three content expert raters.

RESULTS: A total of 72 NAPSAT assessment forms were analyzed. Inter-rater reliability, Cronbach's α = 0.54 and generalizability coefficient = 0.68. The experienced nurses scored higher than the novices, 52.8 versus 62.7, p = 0.009. The provided pass/borderline/fail assessment showed significant difference, p = < 0.001, Cronbach's α = 0.80, with the novices being more likely to fail and the experienced more likely to pass.

CONCLUSION: Assessing sedation skills in a simulator is possible. Video assessment requires expert knowledge of the procedure and the rating matrix. Overall, NAPSAT showed fair inter-rater reliability and good construct validity. This makes NAPSAT fit for formative assessment and proficiency feedback; however, high stakes and summative assessment cannot be advised.