Premium
Application of Latent Trait Models to Identifying Substantively Interesting Raters
Author(s) -
Wolfe Edward W.,
McVay Aaron
Publication year - 2012
Publication title -
educational measurement: issues and practice
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.158
H-Index - 52
eISSN - 1745-3992
pISSN - 0731-1745
DOI - 10.1111/j.1745-3992.2012.00241.x
Subject(s) - trait , centrality , psychology , item response theory , cognitive psychology , rating scale , computer science , latent variable model , association (psychology) , econometrics , latent variable , machine learning , data science , statistics , psychometrics , mathematics , clinical psychology , developmental psychology , programming language , psychotherapist
Historically, research focusing on rater characteristics and rating contexts that enable the assignment of accurate ratings and research focusing on statistical indicators of accurate ratings has been conducted by separate communities of researchers. This study demonstrates how existing latent trait modeling procedures can identify groups of raters who may be of substantive interest to those studying the experiential, cognitive, and contextual aspects of ratings. We employ two data sources in our demonstration—simulated data and data from a large‐scale state‐wide writing assessment. We apply latent trait models to these data to identify examples of rater leniency, centrality, inaccuracy, and differential dimensionality; and we investigate the association between rater training procedures and the manifestation of rater effects in the real data .