Premium
Commentary: A Response to Reckase's Conceptual Framework and Examples for Evaluating Standard Setting Methods
Author(s) -
Schulz E. Matthew
Publication year - 2006
Publication title -
educational measurement: issues and practice
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.158
H-Index - 52
eISSN - 1745-3992
pISSN - 0731-1745
DOI - 10.1111/j.1745-3992.2006.00061.x
Subject(s) - item response theory , psychophysics , computer science , process (computing) , test (biology) , cognitive psychology , psychology , classical test theory , psychometrics , artificial intelligence , perception , clinical psychology , paleontology , neuroscience , biology , operating system
A look at real data shows that Reckase's psychometric theory for standard setting is not applicable to bookmark and that his simulations cannot explain actual differences between methods. It is suggested that exclusively test‐centered, criterion‐referenced approaches are too idealized and that a psychophysics paradigm and a theory of group behavior could be more useful in thinking about the standard setting process. In this view, item mapping methods such as bookmark are reasonable adaptations to fundamental limitations in human judgments of item difficulty. They make item ratings unnecessary and have unique potential for integrating external validity data and student performance data more fully into the standard setting process.