Premium
How faculty members experience workplace‐based assessment rater training: a qualitative study
Author(s) -
Kogan Jennifer R,
Conforti Lisa N,
Bernabeo Elizabeth,
Iobst William,
Holmboe Eric
Publication year - 2015
Publication title -
medical education
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.776
H-Index - 138
eISSN - 1365-2923
pISSN - 0308-0110
DOI - 10.1111/medu.12733
Subject(s) - medical education , psychology , qualitative research , medicine , applied psychology , sociology , social science
Context Direct observation of clinical skills is a common approach in workplace‐based assessment ( WBA ). Despite widespread use of the mini‐clinical evaluation exercise (mini‐ CEX ), faculty development efforts are typically required to improve assessment quality. Little consensus exists regarding the most effective training methods, and few studies explore faculty members' reactions to rater training. Objectives This study was conducted to qualitatively explore the experiences of faculty staff with two rater training approaches – performance dimension training ( PDT ) and a modified approach to frame of reference training (Fo RT ) – to elucidate how such faculty development can be optimally designed. Methods In a qualitative study of a multifaceted intervention using complex intervention principles, 45 out‐patient resident faculty preceptors from 26 US internal medicine residency programmes participated in a rater training faculty development programme. All participants were interviewed individually and in focus groups during and after the programme to elicit how the training influenced their approach to assessment. A constructivist grounded theory approach was used to analyse the data. Results Many participants perceived that rater training positively influenced their approach to direct observation and feedback, their ability to use entrustment as the standard for assessment, and their own clinical skills. However, barriers to implementation and change included: (i) a preference for holistic assessment over frameworks; (ii) challenges in defining competence; (iii) difficulty in changing one's approach to assessment, and (iv) concerns about institutional culture and buy‐in. Conclusions Rater training using PDT and a modified approach to Fo RT can provide faculty staff with assessment skills that are congruent with principles of criterion‐referenced assessment and entrustment, and foundational principles of competency‐based education, while providing them with opportunities to reflect on their own clinical skills. However, multiple challenges to incorporating new forms of training exist. Ongoing efforts to improve WBA are needed to address institutional and cultural contexts, and systems of care delivery.