Premium
The power of the standard test for the presence of heterogeneity in meta‐analysis
Author(s) -
Jackson Dan
Publication year - 2006
Publication title -
statistics in medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.996
H-Index - 183
eISSN - 1097-0258
pISSN - 0277-6715
DOI - 10.1002/sim.2481
Subject(s) - meta analysis , econometrics , statistics , test (biology) , computer science , mathematics , medicine , biology , paleontology
It has been suggested that the standard test for the presence of heterogeneity in meta‐analysis has low power. Although this has been investigated using simulation, there is little direct analytical evidence of the validity of this claim. Using an established approximate distribution for the test statistic, a procedure for obtaining the power of the test is described. From this, a simple formula for the power is obtained. Although this applies to a special case, the formula gives an indication of the power of the test more generally. In particular, for a given significance level, the power can be calibrated in terms of the proportion of the studies' variances that is provided by between‐study variation. A consideration of this quantity confirms that the test does, in general, have low power. It is suggested that practitioners, who wish to conduct the standard test, use the ideas provided in order to investigate the operating characteristics of the test prior to performing it. Copyright © 2005 John Wiley & Sons, Ltd.