Premium
An Enhanced Approach to Combine Item Response Theory With Cognitive Diagnosis in Adaptive Testing
Author(s) -
Wang Chun,
Zheng Chanjin,
Chang HuaHua
Publication year - 2014
Publication title -
journal of educational measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.917
H-Index - 47
eISSN - 1745-3984
pISSN - 0022-0655
DOI - 10.1111/jedm.12057
Subject(s) - weighting , computer science , constraint (computer aided design) , cognition , dual (grammatical number) , data mining , computerized adaptive testing , aggregate (composite) , item response theory , artificial intelligence , machine learning , algorithm , statistics , mathematics , psychology , psychometrics , medicine , materials science , geometry , neuroscience , art , literature , composite material , radiology
Computerized adaptive testing offers the possibility of gaining information on both the overall ability and cognitive profile in a single assessment administration. Some algorithms aiming for these dual purposes have been proposed, including the shadow test approach, the dual information method (DIM), and the constraint weighted method. The current study proposed two new methods, aggregate ranked information index (ARI) and aggregate standardized information index (ASI), which appropriately addressed the noncompatibility issue inherent in the original DIM method. More flexible weighting schemes that put different emphasis on information about general ability (i.e., θ in item response theory) and information about cognitive profile (i.e., α in cognitive diagnostic modeling) were also explored. Two simulation studies were carried out to investigate the effectiveness of the new methods and weighting schemes. Results showed that the new methods with the flexible weighting schemes could produce more accurate estimation of both overall ability and cognitive profile than the original DIM. Among them, the ASI with both empirical and theoretical weights is recommended, and attribute‐level weighting scheme is preferred if some attributes are considered more important from a substantive perspective.