A comparison of recent methods for the analysis of small‐sample cross‐over studies

The standard analysis of variance (ANOVA) method is usually applied to analyse continuous data from cross‐over studies. The method, however, has been known to be not robust for general variance–covariance structure. The simple empirical generalized least squares (EGLS) method, proposed in an attempt to improve the precision of the standard ANOVA method for general variance–covariance structure, is usually insufficient for small‐sample cross‐over trials. In this paper we compare the following commonly used or recent approaches: standard ANOVA; simple EGLS; modified ANOVA method derived from a modified approximate F‐distribution; and a modified EGLS method adjusted by the Kenward and Roger procedure in terms of robustness and power while applying to small‐sample cross‐over studies (say, the sample size is less than 40) over a variety of variance–covariance structures by simulation. We find that the unconditional modified ANOVA method has robust performance for all of the simulated small‐sample cross‐over studies over the various variance–covariance structures, and has comparable power with the standard ANOVA method whenever they are comparable in type I error rate. The EGLS method (simple or modified) is not reliable when the sample size of a cross‐over study is too small, say, less than 24 in the simulation, unless a simple covariance structure is correctly assumed. Given a relatively larger sample size, the modified EGLS method, assuming an unstructured covariance matrix, demonstrates robust performance over the various variance–covariance structures in the simulation and provides more powerful tests than those of the modified (or standard) ANOVA method. Copyright © 2003 John Wiley & Sons, Ltd.