Estimating Classification Decisions for Incomplete Tests

Unforeseen complications during the administration of large‐scale testing programs are inevitable and can prevent examinees from accessing all test material. For classification tests in which the primary purpose is to yield a decision, such as a pass/fail result, the current study investigated a model‐based standard error approach, Bayesian Inference, Binomial Distribution, and Lord–Wingersky Recursion methods to estimate the consistency of making these classification decisions on an incomplete test. Using operational data from a high‐stakes licensure examination, where items are presented in random order, results indicated that all methods were successful in eliminating misclassification when at least half the test was completed. Results from both Binomial and Recursion methods were nearly indistinguishable, yet differences emerged when item sequence was manipulated into difficulty order. Bayesian Inference was the most flexible, relatively unaffected by whether or not the items were randomly presented; however, representative prior data were required, which limits its practical utility. Implications for use in practice, relevant policy decisions, and feasibility for operational implementation are discussed.