Premium
A tool to assess the quality of a meta‐analysis
Author(s) -
Higgins Julian P.T.,
Lane Peter W.,
Anagnostelis Betsy,
AnzuresCabrera Judith,
Baker Nigel F.,
Cappelleri Joseph C.,
Haughie Scott,
Hollis Sally,
Lewis Steff C.,
Moneuse Patrick,
Whitehead Anne
Publication year - 2013
Publication title -
research synthesis methods
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.376
H-Index - 35
eISSN - 1759-2887
pISSN - 1759-2879
DOI - 10.1002/jrsm.1092
Subject(s) - meta analysis , computer science , quality (philosophy) , interpretation (philosophy) , empirical research , management science , data science , statistics , medicine , mathematics , engineering , philosophy , epistemology , programming language
Background Because meta‐analyses are increasingly prevalent and cited in the medical literature, it is important that tools are available to assess their methodological quality. When performing an empirical study of the quality of published meta‐analyses, we found that existing tools did not place a strong emphasis on statistical and interpretational issues. Methods We developed a quality‐assessment tool using existing materials and expert judgment as a starting point, followed by multiple iterations of input from our working group, piloting, and discussion. After having used the tool for our empirical study, agreement for four key items in the tool was measured using weighted kappa coefficients. Results Our tool contained 43 items divided into four key areas (data sources, analysis of individual studies, meta‐analysis methods, and interpretation), and each area ended with a summary question. We also produced guidance for completing the tool. Agreement between raters was fair to moderate. Conclusions The tool should usefully inform subsequent initiatives to develop quality‐assessment tools for meta‐analysis. We advocate use of consensus between independent raters when assessing statistical appropriateness and adequacy of interpretation in meta‐analyses. Copyright © 2013 John Wiley & Sons, Ltd.