Open AccessITEMS BY DESIGN: THE IMPACT OF SYSTEMATIC FEATURE VARIATION ON ITEM STATISTICAL CHARACTERISTICS
Author(s) -
Enright Mary K.,
Morley Mary,
Sheehan Kathleen M.
Publication year - 1999
Publication title -
ets research report series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.235
H-Index - 5
ISSN - 2330-8516
DOI - 10.1002/j.2333-8504.1999.tb01818.x
Subject(s) - variation (astronomy) , feature (linguistics) , item response theory , statistics , sample (material) , sample size determination , psychology , computer science , econometrics , mathematics , psychometrics , linguistics , philosophy , physics , chemistry , chromatography , astrophysics
This study investigated the impact of systematic item feature variation on item statistical characteristics and the degree to which such information could be used as collateral information to supplement examinee performance data and reduce pretest sample size. Two families of word problem variants for the quantitative section of the Graduate Record Examination (GRE ® ) General Test were generated by systematically manipulating item features. For rate problems, the item design features affected item difficulty (Adj. R 2 = .90), item discrimination (Adj. R 2 = .50), and guessing (Adj. R 2 = .41). For probability problems the item design features affected difficulty (Adj. R 2 = .61), but not discrimination or guessing. The results demonstrate the enormous potential of systematically creating item variants. However, questions of how best to manage variants in item pools and to implement statistical procedures that use collateral information must still be resolved.