Open AccessImproving Robustness in Q-Matrix Validation Using an Iterative and Dynamic Procedure
Author(s) -
Pablo Nájera,
Miguel A. Sorrel,
Jimmy de la Torre,
Francisco Abad
Publication year - 2020
Publication title -
applied psychological measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 2.083
H-Index - 64
eISSN - 1552-3497
pISSN - 0146-6216
DOI - 10.1177/0146621620909904
Subject(s) - robustness (evolution) , computer science , matrix (chemical analysis) , cutoff , algorithm , iterative method , context (archaeology) , iterative and incremental development , mathematics , statistics , paleontology , biochemistry , chemistry , materials science , physics , software engineering , quantum mechanics , biology , composite material , gene
In the context of cognitive diagnosis models (CDMs), a Q-matrix reflects the correspondence between attributes and items. The Q-matrix construction process is typically subjective in nature, which may lead to misspecifications. All this can negatively affect the attribute classification accuracy. In response, several methods of empirical Q-matrix validation have been developed. The general discrimination index (GDI) method has some relevant advantages such as the possibility of being applied to several CDMs. However, the estimation of the GDI relies on the estimation of the latent group sizes and success probabilities, which is made with the original (possibly misspecified) Q-matrix. This can be a problem, especially in those situations in which there is a great uncertainty about the Q-matrix specification. To address this, the present study investigates the iterative application of the GDI method, where only one item is modified at each step of the iterative procedure, and the required cutoff is updated considering the new parameter estimates. A simulation study was conducted to test the performance of the new procedure. Results showed that the performance of the GDI method improved when the application was iterative at the item level and an appropriate cutoff point was used. This was most notable when the original Q-matrix misspecification rate was high, where the proposed procedure performed better 96.5% of the times. The results are illustrated using Tatsuoka's fraction-subtraction data set.