Premium
Monitoring Items in Real Time to Enhance CAT Security
Author(s) -
Zhang Jinming,
Li Jie
Publication year - 2016
Publication title -
journal of educational measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.917
H-Index - 47
eISSN - 1745-3984
pISSN - 0022-0655
DOI - 10.1111/jedm.12104
Subject(s) - type i and type ii errors , statistical power , statistics , computer science , cutoff , statistical hypothesis testing , series (stratigraphy) , point (geometry) , statistical analysis , mode (computer interface) , test (biology) , econometrics , reliability engineering , mathematics , engineering , paleontology , physics , geometry , quantum mechanics , biology , operating system
An IRT‐based sequential procedure is developed to monitor items for enhancing test security. The procedure uses a series of statistical hypothesis tests to examine whether the statistical characteristics of each item under inspection have changed significantly during CAT administration. This procedure is compared with a previously developed CTT‐based procedure through simulation studies. The results show that when the total number of examinees is fixed both procedures can control the rate of type I errors at any reasonable significance level by choosing an appropriate cutoff point and meanwhile maintain a low rate of type II errors. Further, the IRT‐based method has a much lower type II error rate or more power than the CTT‐based method when the number of compromised items is small (e.g., 5), which can be achieved if the IRT‐based procedure can be applied in an active mode in the sense that flagged items can be replaced with new items.