Premium
Dealing with effect size multiplicity in systematic reviews and meta‐analyses
Author(s) -
LópezLópez José A.,
Page Matthew J.,
Lipsey Mark W.,
Higgins Julian P.T.
Publication year - 2018
Publication title -
research synthesis methods
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.376
H-Index - 35
eISSN - 1759-2887
pISSN - 1759-2879
DOI - 10.1002/jrsm.1310
Subject(s) - computer science , categorization , meta analysis , systematic review , sample size determination , data science , context (archaeology) , management science , artificial intelligence , mathematics , statistics , medline , medicine , political science , law , economics , paleontology , biology
Systematic reviews often encounter primary studies that report multiple effect sizes based on data from the same participants. These have the potential to introduce statistical dependency into the meta‐analytic data set. In this paper, we provide a tutorial on dealing with effect size multiplicity within studies in the context of meta‐analyses of intervention and association studies, recommending a three‐step approach. The first step is to define the research question and consider the extent to which it mainly reflects interest in mean effect sizes (which we term a convergent approach) or an interest in exploring heterogeneity (which we term a divergent approach). A second step is to identify the types of multiplicities that appear in the initial database of effect sizes relevant to the research question, and we propose a categorization scheme to differentiate them. The third step is to select a strategy for dealing with each type of multiplicity. The researcher can choose between a reductionist meta‐analytic approach, which is characterized by inclusion of a single effect size per study, and an integrative approach, characterized by inclusion of multiple effect sizes per study. We present an overview of available analysis strategies for dealing with effect size multiplicity within studies and provide recommendations intended to help researchers decide which strategy might be preferable in particular situations. Last, we offer caveats and cautions about addressing the challenges multiplicity poses for systematic reviews and meta‐analyses.