Reminder about potentially serious problems with a type of blocked ANOVA analysis

A type of blocked experiment has the potential of being poorly designed and/or analyzed. Verrill and coworkers have referred to such an experiment as a “predictor sort” experiment. David and Gunnink described the procedure as “artificial pairing.” In textbooks it is sometimes referred to as a “matched pair” or “matched subjects” design. The associated design process is also sometimes described as “forming blocks via a concomitant variable.” Improperly designed or analyzed predictor sort experiments can be associated with incorrect or inadequate power calculations and sample sizes, incorrect tests of hypotheses, and incorrect confidence intervals. In this paper we review the main results in the literature, add a section on multiple comparisons, and present results from power and confidence interval coverage simulations that emphasize the importance of the proper design and analysis of predictor sort experiments.