A STUDY OF THE EFFECTS OF ITEM REARRANGEMENT 1

This study was directed to answering the question, “Can the items in a test which is slightly speeded be rearranged in such a way as to discourage cooperation among test takers in a group setting, without affecting the item and score properties of the test?” The literature on the rearrangement problem consists of studies in which the test was either highly speeded or completely unspeeded, or in which rearrangements were extreme. Only moderate rearrangements were used in the present study. Two types of items were used. One group contained 40 items typical of those in the Verbal section of the CEEB Scholastic Aptitude Test (SAT) and one contained 25 items typical of the Mathematics section of the SAT. Four different arrangements of each type of test were administered to spaced samples of a college applicant group in March 1965. Five thousand cases were drawn from each of the eight groups for analysis of total score effects, and independent samples of 495 cases were drawn for item analysis. Results indicated no significant alteration of the item statistics or of the tests' correlations with other tests, but analysis of the regression of the experimental Verbal tests on a longer Verbal test revealed significant differences in intercepts; i.e., some of the Verbal arrangements were slightly more difficult than others. No such difference was found for the Mathematics tests. A possible explanation for these results is that in some of the Verbal arrangements relatively easy items occurred last and were not reached by some students. Thus dropout may have affected test scores differentially. It was concluded that for tests of this type and degree of speededness, an a priori assumption of equivalence among different arrangements might well be false. It was recommended, therefore, that if different arrangements are used in operational administrations of the SAT, separate equating statistics be computed for each arrangement.