Premium
Balancing fit and parsimony to improve Q‐matrix validation
Author(s) -
Nájera Pablo,
Sorrel Miguel A.,
Torre Jimmy,
Abad Francisco José
Publication year - 2021
Publication title -
british journal of mathematical and statistical psychology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.157
H-Index - 51
eISSN - 2044-8317
pISSN - 0007-1102
DOI - 10.1111/bmsp.12228
Subject(s) - hull , computation , matrix (chemical analysis) , flexibility (engineering) , mathematics , point (geometry) , variance (accounting) , computer science , algorithm , domain (mathematical analysis) , measure (data warehouse) , statistics , mathematical optimization , data mining , mathematical analysis , materials science , geometry , accounting , marine engineering , engineering , business , composite material
The Q‐matrix identifies the subset of attributes measured by each item in the cognitive diagnosis modelling framework. Usually constructed by domain experts, the Q‐matrix might contain some misspecifications, disrupting classification accuracy. Empirical Q‐matrix validation methods such as the general discrimination index (GDI) and Wald have shown promising results in addressing this problem. However, a cut‐off point is used in both methods, which might be suboptimal. To address this limitation, the Hull method is proposed and evaluated in the present study. This method aims to find the optimal balance between fit and parsimony, and it is flexible enough to be used either with a measure of item discrimination (the proportion of variance accounted for , PVAF) or a coefficient of determination (pseudo‐ R 2 ). Results from a simulation study showed that the Hull method consistently showed the best performance and shortest computation time, especially when used with the PVAF. The Wald method also performed very well overall, while the GDI method obtained poor results when the number of attributes was high. The absence of a cut‐off point provides greater flexibility to the Hull method, and it places it as a comprehensive solution to the Q‐matrix specification problem in applied settings. This proposal is illustrated using real data.