Premium
Detecting Differential Speededness in Multistage Testing
Author(s) -
Van Der Linden Wim J.,
Breithaupt Krista,
Chuah Siang Chee,
Zhang Yanwei
Publication year - 2007
Publication title -
journal of educational measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.917
H-Index - 47
eISSN - 1745-3984
pISSN - 0022-0655
DOI - 10.1111/j.1745-3984.2007.00030.x
Subject(s) - residual , differential (mechanical device) , set (abstract data type) , time limit , statistics , computer science , test (biology) , probabilistic logic , differential item functioning , mathematics , item response theory , algorithm , psychometrics , paleontology , systems engineering , engineering , biology , programming language , aerospace engineering
A potential undesirable effect of multistage testing is differential speededness, which happens if some of the test takers run out of time because they receive subtests with items that are more time intensive than others. This article shows how a probabilistic response‐time model can be used for estimating differences in time intensities and speed between subtests and test takers and detecting differential speededness. An empirical data set for a multistage test in the computerized CPA Exam was used to demonstrate the procedures. Although the more difficult subtests appeared to have items that were more time intensive than the easier subtests, an analysis of the residual response times did not reveal any significant differential speededness because the time limit appeared to be appropriate. In a separate analysis, within each of the subtests, we found minor but consistent patterns of residual times that are believed to be due to a warm‐up effect, that is, use of more time on the initial items than they actually need.