Premium
Making sense of work‐based assessment: ask the right questions, in the right way, about the right things, of the right people
Author(s) -
Crossley Jim,
Jolly Brian
Publication year - 2012
Publication title -
medical education
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.776
H-Index - 138
eISSN - 1365-2923
pISSN - 0308-0110
DOI - 10.1111/j.1365-2923.2011.04166.x
Subject(s) - judgement , competence (human resources) , psychology , scale (ratio) , applied psychology , reliability (semiconductor) , social psychology , cognitive psychology , political science , law , physics , quantum mechanics , power (physics)
Medical Education 2012: 46: 28–37 Context Historically, assessments have often measured the measurable rather than the important. Over the last 30 years, however, we have witnessed a gradual shift of focus in medical education. We now attempt to teach and assess what matters most. In addition, the component parts of a competence must be marshalled together and integrated to deal with real workplace problems. Workplace‐based assessment (WBA) is complex, and has relied on a number of recently developed methods and instruments, of which some involve checklists and others use judgements made on rating scales. Given that judgements are subjective, how can we optimise their validity and reliability? Methods This paper gleans psychometric data from a range of evaluations in order to highlight features of judgement‐based assessments that are associated with better validity and reliability. It offers some issues for discussion and research around WBA. It refers to literature in a selective way. It does not purport to represent a systematic review, but it does attempt to offer some serious analyses of why some observations occur in studies of WBA and what we need to do about them. Results and Discussion Four general principles emerge: the response scale should be aligned to the reality map of the judges; judgements rather than objective observations should be sought; the assessment should focus on competencies that are central to the activity observed, and the assessors who are best‐placed to judge performance should be asked to participate.