A NEW INDEX OF DIFFERENTIAL SUBGROUP PERFORMANCE: APPLICATION TO THE GRE APTITUDE TEST

The items on the GRE Aptitude Test were analyzed for race (white vs. black) and sex differences in their functioning, using three procedures for identifying items that perform differently: item partial correlations with subgroup membership (race or sex), controlling for total score; comparisons of item characteristic curves for subgroups; and plots of item difficulties for subgroups. All of these methods assume that the test is unidimensional and the total score (or the typical item) is relatively free of differential functioning, and rely on internal analyses of the test to identify deviant items. Appreciable proportions of the items in each section of the test were identified as performing differentially for each race and sex by the partial correlation and ICC indexes, but only small proportions were identified by the item difficulty index. Although many items were identified by the two indexes, these measures were generally very small in magnitude, indicating that they detected a minimal degree of differential functioning. About the same proportions of significant items and the same magnitude of effects occurred in a replication of the analyses in new samples. All of the indexes generally exhibited substantial consistency in identifying the same items in the replicated and original analyses. The partial correlation and ICC indexes substantially agreed in the particular items that they identified as functioning differentially for males and females, but did not agree in the items that they identified as performing differently for whites and blacks; the item difficulty index did not agree with the others in either of these analyses. The items identified by the indexes generally did not differ from the remaining items in type or content, a major exception being that verbal items with female content which were identified by the partial correlation index as performing differently for each sex favored females. The prevalence of differential functioning implied by the present results and the factorial complexity observed in other research brings into question the tenability of the assumptions underlying these indexes. The effect that violating these assumptions has on the findings needs to be determined in further empirical work before the question of the differential performance of these items can be fully answered.