Premium
Competency domains in an undergraduate Objective Structured Clinical Examination: their impact on compensatory standard setting
Author(s) -
Reece Ashley,
Chung Eddie M K,
Gardiner R Mark,
Williams Sian E
Publication year - 2008
Publication title -
medical education
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.776
H-Index - 138
eISSN - 1365-2923
pISSN - 0308-0110
DOI - 10.1111/j.1365-2923.2008.03021.x
Subject(s) - objective structured clinical examination , confirmatory factor analysis , exploratory factor analysis , compensation (psychology) , psychology , medical education , clinical psychology , medicine , statistics , psychometrics , social psychology , structural equation modeling , mathematics
Context Following a 15‐week attachment in paediatrics and child health, general practice and dermatology medical students in their second clinical year at this medical school undertake a high‐stakes assessment including an objective structured clinical examination (OSCE). There were 2 hypotheses. Firstly, groups of similar stations map to competency domains identifiable by factor analysis. Secondly, poor performance in individual domains is compensated for by achieving the required standard of performance across the whole assessment. Methods A total of 647 medical students were assessed by an OSCE during 5 individual examination sittings (diets) over 2 years. Ten scoring stations in the OSCE were analysed and confirmatory factor analysis performed comparing a 1‐factor model (where all the stations are discrete entities related to one underlying domain) with a 3‐factor model (where the stations load onto 3 domains from a previously reported exploratory factor analysis). Results The 3‐factor model yielded a significantly better fit to the data (χ 2 = 15.3, P < 0.01). Assessing the compensation data of 1 diet, 29 of 127 students failed in 1 or more domains described, whereas only 5 failed if compensation was allowed across all domains. Discussion Confirmatory factor analysis showed a significant fit of the data to previously described competency domains for a high‐stakes undergraduate OSCE. Compensation within but not between competency domains would provide a more robust standard, improve validity, and substantially reduce the pass rate.