Open AccessBias in meta‐analyses using Hedges’ d
Author(s) -
Hamman Elizabeth A.,
Pappalardo Paula,
Bence James R.,
Peacor Scott D.,
Osenberg Craig W.
Publication year - 2018
Publication title -
ecosphere
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.255
H-Index - 57
ISSN - 2150-8925
DOI - 10.1002/ecs2.2419
Subject(s) - statistics , weighting , sample size determination , meta analysis , replication (statistics) , metric (unit) , mathematics , variance (accounting) , contrast (vision) , confidence interval , publication bias , econometrics , sample (material) , inverse , computer science , physics , artificial intelligence , medicine , operations management , accounting , business , geometry , acoustics , economics , thermodynamics
Abstract The type of metric and weighting method used in meta‐analysis can create bias and alter coverage of confidence intervals when the estimated effect size and its weight are correlated. Here, we investigate bias associated with the common metric, Hedges’ d , under conditions common in ecological meta‐analyses. We simulated data from experiments, computed effect sizes and their variances, and performed meta‐analyses applying three weighting schemes (inverse variance, sample size, and unweighted) for varying levels of effect size, within‐study replication, number of studies in the meta‐analysis, and among‐study variance. Unweighted analyses, and those using weights based on sample size, were close to unbiased and yielded coverages close to the nominal level of 0.95. In contrast, the inverse‐variance weighting scheme led to bias and low coverage, especially for meta‐analyses based on studies with low replication. This bias arose because of a correlation between the estimated effect and its weight when using the inverse‐variance method. In many cases, the sample size weighting scheme was most efficient, and, when not, the differences in efficiency among the three methods were relatively minor. Thus, if using Hedges’ d , we recommend using weights based upon sample size that do not involve individual study estimates of the effect size.