Open AccessA Signal Detection Model for Multiple-Choice Exams
Author(s) -
Lawrence T. DeCarlo
Publication year - 2021
Publication title -
applied psychological measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 2.083
H-Index - 64
eISSN - 1552-3497
pISSN - 0146-6216
DOI - 10.1177/01466216211014599
Subject(s) - choice set , multinomial logistic regression , multinomial distribution , set (abstract data type) , extension (predicate logic) , multiple choice , computer science , two alternative forced choice , detection theory , logit , econometrics , signal (programming language) , item response theory , statistics , artificial intelligence , mathematics , machine learning , psychometrics , significant difference , telecommunications , detector , programming language
A model for multiple-choice exams is developed from a signal-detection perspective. A correct alternative in a multiple-choice exam can be viewed as being a signal embedded in noise (incorrect alternatives). Examinees are assumed to have perceptions of the plausibility of each alternative, and the decision process is to choose the most plausible alternative. It is also assumed that each examinee either knows or does not know each item. These assumptions together lead to a signal detection choice model for multiple-choice exams. The model can be viewed, statistically, as a mixture extension, with random mixing, of the traditional choice model, or similarly, as a grade-of-membership extension. A version of the model with extreme value distributions is developed, in which case the model simplifies to a mixture multinomial logit model with random mixing. The approach is shown to offer measures of item discrimination and difficulty, along with information about the relative plausibility of each of the alternatives. The model, parameters, and measures derived from the parameters are compared to those obtained with several commonly used item response theory models. An application of the model to an educational data set is presented.