Premium
Using Diagnostic Profiles to Describe Borderline Performance in Standard Setting
Author(s) -
Skaggs Gary,
Hein Serge F.,
Wilkins Jesse L. M.
Publication year - 2018
Publication title -
educational measurement: issues and practice
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.158
H-Index - 52
eISSN - 1745-3992
pISSN - 0731-1745
DOI - 10.1111/emip.12228
Subject(s) - test (biology) , task (project management) , strengths and weaknesses , psychology , computer science , statistics , applied psychology , social psychology , mathematics , engineering , paleontology , systems engineering , biology
In test‐centered standard‐setting methods, borderline performance can be represented by many different profiles of strengths and weaknesses. As a result, asking panelists to estimate item or test performance for a hypothetical group study of borderline examinees, or a typical borderline examinee, may be an extremely difficult task and one that can lead to questionable results in setting cut scores. In this study, data collected from a previous standard‐setting study are used to deduce panelists’ conceptions of profiles of borderline performance. These profiles are then used to predict cut scores on a test of algebra readiness. The results indicate that these profiles can predict a very wide range of cut scores both within and between panelists. Modifications are proposed to existing training procedures for test‐centered methods that can account for the variation in borderline profiles.