Assessing Error in Behavioral Data: Problems of Sequencing

Both classical test theory and generalizability theory focus on measurement error as a group property. Thus, common estimates o f errors o f measurement are developed for all members o f a group. But with behavioral data, unlike specifically test data, it is sometimes possible to estimate error separately for each individual. This enables one to ask questions about the relationships between error o f measurement and other characteristics o f the individual. Consequently, it also makes possible the use of regression techniques to call upon group data to improve the estimates o f individual measurement error. The focus on the individual also lays bare the possibility o f sequencing effects, and it is shown that, even in the absence o f trend, autocorrelation can cause standard procedures to grossly underestimate the magnitude o f measurement error. Classroom observation data are examined for autocorrelation, and recommendations are made about the scheduling o f data collection so as to minimize its effects.