Premium
Hierarchical Logistic Regression: Accounting for Multilevel Data in DIF Detection
Author(s) -
French Brian F.,
Finch W. Holmes
Publication year - 2010
Publication title -
journal of educational measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.917
H-Index - 47
eISSN - 1745-3984
pISSN - 0022-0655
DOI - 10.1111/j.1745-3984.2010.00115.x
Subject(s) - type i and type ii errors , logistic regression , statistics , multilevel model , cluster (spacecraft) , variable (mathematics) , differential item functioning , hierarchical database model , statistical power , hierarchical clustering , regression analysis , regression , computer science , econometrics , item response theory , data mining , mathematics , psychometrics , cluster analysis , mathematical analysis , programming language
The purpose of this study was to examine the performance of differential item functioning (DIF) assessment in the presence of a multilevel structure that often underlies data from large‐scale testing programs. Analyses were conducted using logistic regression (LR), a popular, flexible, and effective tool for DIF detection. Data were simulated using a hierarchical framework, such as might be seen when examinees are clustered in schools, for example. Both standard and hierarchical LR (accounting for multilevel data) approaches to DIF detection were employed. Results highlight the differences in DIF detection rates when the analytic strategy matches the data structure. Specifically, when the grouping variable was within clusters, LR and HLR performed similarly in terms of Type I error control and power. However, when the grouping variable was between clusters, LR failed to maintain the nominal Type I error rate of .05. HLR was able to maintain this rate. However, power for HLR tended to be low under many conditions in the between cluster variable case.