Premium
The Answer Key as a Source of Error in Examinations for Professionals
Author(s) -
Norcini John J.
Publication year - 1987
Publication title -
journal of educational measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.917
H-Index - 47
eISSN - 1745-3984
pISSN - 0022-0655
DOI - 10.1111/j.1745-3984.1987.tb00283.x
Subject(s) - certification , competence (human resources) , psychology , grading (engineering) , key (lock) , medical education , computer science , applied psychology , mathematics education , social psychology , medicine , political science , engineering , civil engineering , computer security , law
There has been a growing emphasis on written examinations that assess the ability of physicians and teachers to make decisions in the absence of protocols for action—a crucial aspect of professional competence. A characteristic of such tests is controversy, even among experts, about what constitutes the correct response to some of the items. This paper studied the impact of variability in answer keys, constructed using the aggregate method, on total errors of measurement. Results indicated that several scorers made a sizable contribution to reduction in measurement error and that scorers or groups of scorers who each developed the answer key for a subset of items produced better results than a single group that developed the answer key for all items. Implications of judgment tests scored using the aggregate method are described for teacher and physician certification.