Open AccessCOMPUTERIZED MASTERY TESTING WITH NONEQUIVALENT TESTLETS *
Author(s) -
Sheehan Kathleen,
Lewis Charles
Publication year - 1990
Publication title -
ets research report series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.235
H-Index - 5
ISSN - 2330-8516
DOI - 10.1002/j.2333-8504.1990.tb01352.x
Subject(s) - set (abstract data type) , statistics , permutation (music) , certification , test (biology) , psychology , mathematics , econometrics , computer science , paleontology , physics , acoustics , political science , law , biology , programming language
A practical procedure for determining the effect of testlet nonequivalence on the operating characteristics of a testlet‐based computerized mastery test (CMT) is introduced. The procedure involves estimating the CMT decision rule twice, once with testlets treated as equivalent and once with testlets treated as nonequivalent. In the equivalent testlet mode, the likelihood functions estimated for specific number right scores are assumed to be constant across testlets and a single set of cutscores is estimated for all testlets. In the nonequivalent testlet mode, the likelihood functions estimated for specific number‐right scores are allowed to vary from one testlet to another and a different set of cutscores is estimated for each permutation of testlet presentation order. Small differences between the estimated operating characteristics of the equivalent testlet decision rule and the nonequivalent testlet decision rule indicate that the assumption of equivalent testlets was warranted. This procedure is illustrated with data from the Architect Registration Examination, a professional certification examination administered by the National Council of Architectural Registration Boards (NCARB).