Premium
Classification Consistency and Accuracy With Atypical Score Distributions
Author(s) -
Kim Stella Y.,
Lee WonChan
Publication year - 2019
Publication title -
journal of educational measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.917
H-Index - 47
eISSN - 1745-3984
pISSN - 0022-0655
DOI - 10.1111/jedm.12250
Subject(s) - multinomial distribution , bimodality , consistency (knowledge bases) , statistics , mathematics , distribution (mathematics) , sample (material) , econometrics , mathematical analysis , physics , geometry , quantum mechanics , galaxy , thermodynamics
The current study aims to evaluate the performance of three non‐IRT procedures (i.e., normal approximation, Livingston‐Lewis, and compound multinomial) for estimating classification indices when the observed score distribution shows atypical patterns: (a) bimodality, (b) structural (i.e., systematic) bumpiness, or (c) structural zeros (i.e., no frequencies). Under a bimodal distribution, the normal approximation procedure produced substantially large bias. For a distribution with structural bumpiness, the compound multinomial procedure tended to introduce larger bias. Under a distribution with structural zeroes, the relative performance of selected estimation procedures depended on cut score location and the sample‐size conditions. In general, the differences in estimation errors among the three procedures were not substantially large.