Premium
Simulation Studies of the Effects of Small Sample Size and Studied Item Parameters on SIBTEST and Mantel‐Haenszel Type I Error Performance
Author(s) -
Roussos Louis A.,
Stout William F.
Publication year - 1996
Publication title -
journal of educational measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.917
H-Index - 47
eISSN - 1745-3984
pISSN - 0022-0655
DOI - 10.1111/j.1745-3984.1996.tb00490.x
Subject(s) - type i and type ii errors , statistics , rasch model , differential item functioning , sample size determination , sample (material) , item response theory , mathematics , econometrics , psychology , psychometrics , chemistry , chromatography
Two simulation studies investigated Type I error performance of two statistical procedures for detecting differential item functioning (DIF): SIBTEST and Mantel‐Haenszel (MH). Because MH and SIBTEST are based on asymptotic distributions requiring “large” numbers of examinees, the first study examined Type 1 error for small sample sizes. No significant Type I error inflation occurred for either procedure. Because MH has the potential for Type I error inflation for non‐Rasch models, the second study used a markedly non‐Rasch test and systematically varied the shape and location of the studied item. When differences in distribution across examinee group of the measured ability were present, both procedures displayed inflated Type 1 error for certain items; MH displayed the greater inflation. Also, both procedures displayed statistically biased estimation of the zero DIF for certain items, though SIBTEST displayed much less than MH. When no latent distributional differences were present, both procedures performed satisfactorily under all conditions.