ANALYSIS OF ITEM RESPONSE PATTERNS. QUESTIONABLE TEST DATA AND DISSIMILAR CURRICULUM PRACTICES

It has been common practice to simply use a single total score although some attempts have been made to use wrong response patterns diagnostically. This is partially attributable to the fact that there are limited amounts of data available for particular response patterns, and because there is both low reliability of individual items and a tendency to attribute a large fraction of the common variance of the items to a single dimension. Recently, however, several approaches have been suggested which are intended to identify unusual response patterns. There are a wide variety of factors that could lead to an unusual response pattern. While a question may be very easy for most students, unique background experiences may make that same item very difficult for others. For example, the child who has never gone camping may find a reading passage about a camping experience more difficult than children who have had the experience. Other individual differences in motivational disposition, for instance, test anxiety, may make a normally simple item very difficult for some people. Students' exposure to different subject matter, and the way in which that subject matter has been stressed, will influence how they perform on tests. This may result in some measurable variation in scores from class to class. Items which are generally difficult for most students may be relatively easy for students who have been in classes where that particular content was emphasized. Such variation from the norm may lead to the systematic over- or under-estimation of an individual's or group's level of achievement, distorting the measurement results. Indices measuring the degree to which the response pattern for an individual is unusual could be used in a variety of ways. They could identify individuals for whom the standard interpretation of the test score is misleading, or identify groups with atypical instructional and/or experiential histories that alter the relative difficulty ordering of the items. In addition, the items that contribute most to high values on an index for particular subgroups could be identified and judgments made regarding the appropriateness of the item content for those subgroups. There are two major types of indices of the degree to which an individual's pattern of responses is unusual. First, there are the "appropriateness" indices which are based upon