Premium
Modeling Randomness in Judging Rating Scales with a Random‐Effects Rating Scale Model
Author(s) -
Wang WenChung,
Wilson Mark,
Shih ChingLin
Publication year - 2006
Publication title -
journal of educational measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.917
H-Index - 47
eISSN - 1745-3984
pISSN - 0022-0655
DOI - 10.1111/j.1745-3984.2006.00020.x
Subject(s) - response surface methodology , statistics , mathematics , random effects model , randomness , reliability (semiconductor) , medicine , power (physics) , meta analysis , physics , quantum mechanics
This study presents the random‐effects rating scale model (RE‐RSM) which takes into account randomness in the thresholds over persons by treating them as random‐effects and adding a random variable for each threshold in the rating scale model (RSM) ( Andrich, 1978 ). The RE‐RSM turns out to be a special case of the multidimensional random coefficients multinomial logit model (MRCMLM) ( Adams, Wilson, & Wang, 1997 ) so that the estimation procedures for the MRCMLM can be directly applied. The results of the simulation indicated that when the data were generated from the RSM, using the RSM and the RE‐RSM to fit the data made little difference: both resulting in accurate parameter recovery. When the data were generated from the RE‐RSM, using the RE‐RSM to fit the data resulted in unbiased estimates, whereas using the RSM resulted in biased estimates, large fit statistics for the thresholds, and inflated test reliability. An empirical example of 10 items with four‐point rating scales was illustrated in which four models were compared: the RSM, the RE‐RSM, the partial credit model ( Masters, 1982 ), and the constrained random‐effects partial credit model. In this real data set, the need for a random‐effects formulation becomes clear.