Premium
The effect of candidate familiarity on examiner OSCE scores
Author(s) -
Jefferies Ann,
Simmons Brian,
Regehr Glenn
Publication year - 2007
Publication title -
medical education
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.776
H-Index - 138
eISSN - 1365-2923
pISSN - 0308-0110
DOI - 10.1111/j.1365-2923.2007.02820.x
Subject(s) - checklist , cronbach's alpha , medicine , rating scale , objective structured clinical examination , psychology , family medicine , clinical psychology , medical education , psychometrics , developmental psychology , cognitive psychology
Context Although examiners are a large source of variability in the objective structured clinical examination (OSCE), the exact causes of examiner variance remain understudied. Objective This study aimed to determine whether examiner familiarity with candidates influences candidate scores. Methods A total of 24 candidates from 4 neonatal‐perinatal training programmes participated in a 10‐station OSCE. Sixteen trainees and 7 examiners came from a single centre (site A) and 8 candidates and 5 examiners came from the other 3 centres. Examiners completed station‐specific binary checklists and an overall global rating; standardised patients (SPs) and standardised health professionals (SHPs) completed 4 process ratings and the overall rating. A fixed‐effect, 2‐way analysis of variance was performed to ascertain whether there was interaction between examiner site and candidate site. Results Interstation Cronbach's α was 0.80 for the examiner checklist, 0.88 for the examiner global rating and 0.88 for the SP or SHP global rating. Although the checklist scores awarded by site A examiners were significantly higher than those awarded by non‐site A examiners, there was no significant interaction between examiner and candidate site ( P = 0.124). Similarly, the interaction between examiner and candidate site for the global rating was not significant ( P = 0.207). Global ratings awarded by SPs and SHPs were also higher in stations where site A faculty examined site A candidates, suggesting the observed differences may have been related to performance. Conclusions Results from this small dataset suggest that examiner familiarity with candidates does not influence how examiners score candidates, confirming the objective nature of the OSCE. Confirmation with a larger study is required.