Open AccessA STUDY OF CONFIDENCE AND ACCURACY USING THE RASCH MODELING PROCEDURES
Author(s) -
Paek Insu,
Lee Jihyun,
Stankov Lazar,
Wilson Mark
Publication year - 2008
Publication title -
ets research report series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.235
H-Index - 5
ISSN - 2330-8516
DOI - 10.1002/j.2333-8504.2008.tb02128.x
Subject(s) - rasch model , overconfidence effect , confidence interval , dimension (graph theory) , statistics , psychology , test (biology) , econometrics , mathematics , social psychology , paleontology , biology , pure mathematics
This study investigated the relationship between students' actual performance (accuracy) and their subjective judgments of accuracy (confidence) on selected English language proficiency tests. The unidimensional and multidimensional IRT Rasch approaches were used to model the discrepancy between confidence and accuracy at the item and test level and to assess disattenuated strength of association between accuracy and confidence. The analysis results indicate a pattern of overconfidence bias (i.e., overestimation of success rate), which was related to item difficulty. In addition, the strength of association between accuracy and confidence dimension was relatively high: The confidence dimension explained 45% and 52% of the variability in the accuracy dimension for the two tests employed in this study.