Premium
Using Response Time to Detect Item Preknowledge in Computer‐Based Licensure Examinations
Author(s) -
Qian Hong,
Staniewska Dorota,
Reckase Mark,
Woo Ada
Publication year - 2016
Publication title -
educational measurement: issues and practice
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.158
H-Index - 52
eISSN - 1745-3992
pISSN - 0731-1745
DOI - 10.1111/emip.12102
Subject(s) - licensure , item response theory , sample (material) , test (biology) , computerized adaptive testing , item bank , psychology , item analysis , computer science , feature (linguistics) , psychometrics , medicine , medical education , clinical psychology , paleontology , linguistics , chemistry , philosophy , chromatography , biology
This article addresses the issue of how to detect item preknowledge using item response time data in two computer‐based large‐scale licensure examinations. Item preknowledge is indicated by an unexpected short response time and a correct response. Two samples were used for detecting item preknowledge for each examination. The first sample was from the early stage of the operational test and was used for item calibration. The second sample was from the late stage of the operational test, which may feature item preknowledge. The purpose of this research was to explore whether there was evidence of item preknowledge and compromised items in the second sample using the parameters estimated from the first sample. The results showed that for one nonadaptive operational examination, two items (of 111) were potentially exposed, and two candidates (of 1,172) showed some indications of preknowledge on multiple items. For another licensure examination that featured computerized adaptive testing, there was no indication of item preknowledge or compromised items. Implications for detected aberrant examinees and compromised items are discussed in the article.