Open AccessDIF DETECTION WITH SMALL SAMPLES: APPLYING SMOOTHING TECHNIQUES TO FREQUENCY DISTRIBUTIONS IN THE MANTEL‐HAENSZEL PROCEDURE
Author(s) -
Yu Lei,
Moses Tim,
Puhan Gautam,
Dorans Neil
Publication year - 2008
Publication title -
ets research report series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.235
H-Index - 5
ISSN - 2330-8516
DOI - 10.1002/j.2333-8504.2008.tb02130.x
Subject(s) - smoothing , statistics , differential item functioning , mathematics , sample size determination , type i and type ii errors , sample (material) , population , data set , item response theory , psychometrics , chemistry , demography , chromatography , sociology
All differential item functioning (DIF) methods require at least a moderate sample size for effective DIF detection. Samples that are less than 200 pose a challenge for DIF analysis. Smoothing can improve upon the estimation of the population distribution by preserving major features of an observed frequency distribution while eliminating the noise brought about by irregular data points. This study applied smoothing techniques to frequency distributions and investigated the impact of smoothed data on the Mantel‐Haenszel (MH) DIF detection in small samples. Eight sample‐size combinations were randomly drawn from a real data set to make the study realistic and were replicated 80 times to produce stable results. The population DIF results were used as the criteria to evaluate sample estimates using root‐mean square difference (RMSD), bias analysis, and Type II error rate. Loglinear smoothing was found to provide slight to moderate improvements in MH DIF estimation with small samples.