Generalization Variables and Restricted Hypotheses

The first half of the paper, Generalization Variables, points out a serious flaw in the statistical foundations of much research in reading, namely, some researchers have neglected to consider whether their findings could be replicated using a different sample of materials. Since materials frequently interact with treatments, this neglect is responsible for many of the conflicting findings in the field. Several examples are given and used to illustrate appropriate significance tests. The second half, Restricted Hypotheses, shows that many topics in reading research do not quite fit the model for the null hypothesis. That is, the hypothesis can be neither rejected nor accepted in all or none fashion; instead, it is restricted, true under some conditions, false under others. The most effective strategy is not to risk all resources on a single experiment that tests a null hypothesis, but to home in on these restrictions with a systematic series of converging experiments.