Premium
Evaluating structured assessment of anaesthesiologists' non‐technical skills
Author(s) -
Jepsen R. M. H. G.,
Dieckmann P.,
Spanager L.,
LykJensen H. T.,
Konge L.,
Ringsted C.,
Østergaard D.
Publication year - 2016
Publication title -
acta anaesthesiologica scandinavica
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.738
H-Index - 107
eISSN - 1399-6576
pISSN - 0001-5172
DOI - 10.1111/aas.12709
Subject(s) - medicine , reliability (semiconductor) , session (web analytics) , applied psychology , medical education , psychology , computer science , world wide web , power (physics) , physics , quantum mechanics
Background Non‐technical skills ( NTS ) are essential for safe and efficient anaesthesia. Assessment instruments with appropriate validity evidence can be used to ensure that anaesthesiologists possess the NTS necessary to deliver high‐standard patient care. The aims were to collect validity evidence using a contemporary validity framework for the assessment instrument Anaesthesiologists’ Non‐Technical Skills in Denmark ( ANTS dk) regarding response process and internal structure (including reliability), and to investigate the effect of rater training on these properties. Methods An explorative study was undertaken at the Danish Institute for Medical Simulation, Copenhagen, Denmark. In a 1‐day session, using ANTS dk, a convenience sample of 19 anaesthesiologists rated trainee anaesthesiologists’ NTS in nine video‐recorded simulation scenarios before and after a 3‐h training session. Results Response process evidence: participants considered ANTS dk useful and feasible for NTS assessment. Internal structure evidence: inter‐rater reliability (single measures) largely expressed substantial agreement ( ICC ≥ 0.55 and ICC ≥ 0.60 for pre‐ and post‐training ratings respectively). Strong internal consistency of ratings was found (Spearman's correlation coefficient ≥ 0.82). Accuracy of participants’ ratings compared with reference ratings (± 1 scale point) was notable (76% and 78% for pre‐ and post‐training ratings, respectively). The results indicate that the elements ‘Demonstrating self‐awareness’, ‘Reassessing decisions’, ‘Assessing competencies’, and ‘Supporting others’ need more attention in future rater training. Conclusion The validity evidence collected on content, response process, and internal structure, suggests that ANTS dk is easy to use on video‐recorded simulation scenarios, indicating that ANTS dk is a feasible instrument for NTS assessment during anaesthesia training.