Assessing Testlet Effect, Impact, Differential Testlet, and Item Functioning Using Cross-Classified Multilevel Measurement Modeling

The present study used the two-level testlet response model(MMMT-2) to assess impact, differential item functioning (DIF), and differential testletfunctioning (DTLF) in a reading comprehension test. The data came from 21,641 applicantsinto English Masters’ programs at Iranian state universities. Testlet effects wereestimated, and items and testlets that were functioning differentially for test takersof different genders and majors were identified. Also parameter estimates obtained underMMMT-2 and those obtained under the two-level hierarchical generalized linear model(HGLM-2) were compared. The results indicated that ability estimates obtained under thetwo models were significantly different at the lower and upper ends of the abilitydistribution. In addition, it was found that ignoring local item dependence (LID) wouldresult in overestimation of the precision of the ability estimates. As for thedifficulty of the items, the estimates obtained under the two models were almost thesame, but standard errors were significantly different