An evaluation of methods for the stratified analysis of clustered binary data in community intervention trials

A simulation study is conducted in a community intervention setting. Several methods of stratified analysis of clustered binary data are compared in terms of empirical significance and empirical power levels. They are the Mantel–Haenszel test statistic (χ 2 MH ), the adjusted Mantel–Haenszel test statistic of Donald–Donner (χ 2 DD ), Rao–Scott (χ 2 RSN and χ 2 RSP ), and Zhang–Boos (χ 2 ZBN and χ 2 ZBP ), Wald (χ 2 W ), robust Wald (χ 2 RW ), score (χ 2 S ), robust score (χ 2 RS ), and the test statistic based on generalized linear mixed model (GLMM) (χ 2 GLMM ). When ρ ≠ 0, χ 2 MH has inflated type I error, and it should not be used when observations are correlated. The results also warn of the use of χ 2 RSN and χ 2 RW due to their poor performance in terms of empirical significance level. χ 2 ZBP and χ 2 GLMM have better empirical significance levels as compared to other statistics; however, χ 2 ZBP tends to have lower empirical powers than other statistics when the number of clusters ( N ) is less than 24. χ 2 RSP provides the highest empirical powers when ρ ≥ 0.1 and N ≤ 12. When ρ ≤ 0.01, we recommend the use of χ 2 RS and χ 2 GLMM since they have better overall performance in terms of empirical significance levels and empirical power levels. Copyright © 2003 John Wiley & Sons, Ltd.