Premium
Exploring the use of rating scales with entrustment anchors in workplace‐based assessment
Author(s) -
Robinson Trevor J. G.,
Wagner Natalie,
Szulewski Adam,
Dudek Nancy,
Cheung Warren J.,
Hall Andrew K.
Publication year - 2021
Publication title -
medical education
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.776
H-Index - 138
eISSN - 1365-2923
pISSN - 0308-0110
DOI - 10.1111/medu.14573
Subject(s) - competence (human resources) , rating scale , medical education , context (archaeology) , psychology , medicine , nursing , social psychology , developmental psychology , paleontology , biology
Purpose Competency‐based medical education (CBME) has prompted widespread implementation of workplace‐based assessment (WBA) tools using entrustment anchors. This study aimed to identify factors that influence faculty's rating choices immediately following assessment and explore their experiences using WBAs with entrustment anchors, specifically the Ottawa Surgical Competency Operating Room Evaluation scale. Method A convenience sample of 50 semi‐structured interviews with Emergency Medicine (EM) physicians from a single Canadian hospital were conducted between July and August 2019. All interviews occurred within two hours of faculty completing a WBA of a trainee. Faculty were asked what they considered when rating the trainee's performance and whether they considered an alternate rating. Two team members independently analysed interview transcripts using conventional content analysis with line‐by‐line coding to identify themes. Results Interviews captured interactions between 70% (26/37) of full‐time EM faculty and 86% (19/22) of EM trainees. Faculty most commonly identified the amount of guidance the trainee required as influencing their rating. Other variables such as clinical context, trainee experience, past experiences with the trainee, perceived competence and confidence were also identified. While most faculty did not struggle to assign ratings, some had difficulty interpreting the language of entrustment anchors, being unsure whether their assessment should be retrospective or prospective in nature, and if/how the assessment should change whether they were ‘in the room’ or not. Conclusions By going to the frontline during WBA encounters, this study captured authentic and honest reflections from physicians immediately engaged in assessment using entrustment anchors. While many of the factors identified are consistent with previous retrospective work, we highlight how some faculty consider factors outside the prescribed approach and struggle with the language of entrustment anchors. These results further our understanding of ‘in‐the‐moment’ assessments using entrustment anchors and may facilitate effective faculty development regarding WBA in CBME.