Premium
The problem of measurement bias in comparing selected subgroups
Author(s) -
Mendoza Jorge L.,
Lee Seunghoo,
Fife Dustin
Publication year - 2021
Publication title -
british journal of mathematical and statistical psychology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.157
H-Index - 51
eISSN - 2044-8317
pISSN - 0007-1102
DOI - 10.1111/bmsp.12215
Subject(s) - statistics , estimator , mathematics , bias of an estimator , selection (genetic algorithm) , selection bias , variance (accounting) , population , reliability (semiconductor) , econometrics , minimum variance unbiased estimator , demography , computer science , economics , power (physics) , physics , accounting , quantum mechanics , artificial intelligence , sociology
Estimates of subgroup differences are routinely used as part of a comprehensive validation system, and these estimates serve a critical role, including evaluating adverse impact. Unfortunately, under direct range restriction, a selected mean ( μ ̂ t ′ ) is a biased estimator of the population mean μ x as well as the selected true score mean μ t ′ . This is due partly to measurement bias. This bias, as we show, is a factor of the selection ratio, the reliability of the measure, and the variance of the distribution. This measurement bias renders a subgroup comparison questionable when the subgroups have different selection ratios. The selected subgroup comparison is further complicated by the fact that the subgroup variances will be unequal in most situations where the selection ratios are not equal. We address these problems and present a corrected estimate of the mean difference, as well as an estimate of Cohen’s d* that estimates the true score difference between two selected populations,( μ tA ′ - μ tB ′ ) / σ t . In addition, we show that the measurement bias is not present under indirect range restriction. Thus, the observed selected meanμ ̂ t ′ is an unbiased estimator of selected true score mean μ ty ′ . However, it is not an unbiased estimator of the population mean μ y . These results have important implications for selection research, particularly when validating instruments.