Premium
Evaluating the Comparability of Paper‐ and Computer‐Based Science Tests Across Sex and SES Subgroups
Author(s) -
Randall Jennifer,
Sireci Stephen,
Li Xueming,
Kaira Leah
Publication year - 2012
Publication title -
educational measurement: issues and practice
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.158
H-Index - 52
eISSN - 1745-3992
pISSN - 0731-1745
DOI - 10.1111/j.1745-3992.2012.00252.x
Subject(s) - comparability , differential item functioning , rasch model , test (biology) , measurement invariance , psychology , item response theory , construct (python library) , licensure , construct validity , certification , confirmatory factor analysis , applied psychology , statistics , psychometrics , clinical psychology , social psychology , developmental psychology , computer science , mathematics , medical education , medicine , structural equation modeling , paleontology , combinatorics , political science , law , biology , programming language
As access and reliance on technology continue to increase, so does the use of computerized testing for admissions, licensure/certification, and accountability exams. Nonetheless, full computer‐based test (CBT) implementation can be difficult due to limited resources. As a result, some testing programs offer both CBT and paper‐based test (PBT) administration formats. In such situations, evidence that scores obtained from different formats are comparable must be gathered. In this study, we illustrate how contemporary statistical methods can be used to provide evidence regarding the comparability of CBT and PBT scores at the total test score and item levels. Specifically, we looked at the invariance of test structure and item functioning across test administration mode across subgroups of students defined by SES and sex. Multiple replications of both confirmatory factor analysis and Rasch differential item functioning analyses were used to assess invariance at the factorial and item levels. Results revealed a unidimensional construct with moderate statistical support for strong factorial‐level invariance across SES subgroups, and moderate support of invariance across sex. Issues involved in applying these analyses to future evaluations of the comparability of scores from different versions of a test are discussed.