Open AccessSTUDIES OF A LATENT‐CLASS SIGNAL‐DETECTION MODEL FOR CONSTRUCTED‐RESPONSE SCORING
Author(s) -
DeCarlo Lawrence T.
Publication year - 2008
Publication title -
ets research report series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.235
H-Index - 5
ISSN - 2330-8516
DOI - 10.1002/j.2333-8504.2008.tb02149.x
Subject(s) - rubric , latent class model , computer science , grading (engineering) , class (philosophy) , artificial intelligence , probabilistic latent semantic analysis , latent variable , pattern recognition (psychology) , machine learning , statistics , data mining , psychology , mathematics , mathematics education , civil engineering , engineering
Rater behavior in essay grading can be viewed as a signal‐detection task, in that raters attempt to discriminate between latent classes of essays, with the latent classes being defined by a scoring rubric. The present report examines basic aspects of an approach to constructed‐response (CR) scoring via a latent‐class signal‐detection model. The model provides a psychological framework for CR scoring and includes rater parameters with a clear cognitive basis. Simulations are used to examine how well rater parameters and latent‐class sizes are recovered as well as the accuracy of classification. The relation of rater parameters to agreement statistics and classification accuracy is examined. The effects of using a balanced, incomplete block design are compared to those for a fully crossed design. The model is applied to several ETS datasets.