Premium
Missing binary data extraction challenges from Cochrane reviews in mental health and Campbell reviews with implications for empirical research
Author(s) -
Spineli Loukia M.
Publication year - 2017
Publication title -
research synthesis methods
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.376
H-Index - 35
eISSN - 1759-2887
pISSN - 1759-2879
DOI - 10.1002/jrsm.1268
Subject(s) - missing data , meta analysis , data extraction , systematic review , computer science , outcome (game theory) , statistics , medline , mental health , data collection , psychology , data mining , medicine , psychiatry , machine learning , mathematics , mathematical economics , political science , law
Objectives Tο report challenges encountered during the extraction process from Cochrane reviews in mental health and Campbell reviews and to indicate their implications on the empirical performance of different methods to handle missingness. Methods We used a collection of meta‐analyses on binary outcomes collated from a previous work on missing outcome data. To evaluate the accuracy of their extraction, we developed specific criteria pertaining to the reporting of missing outcome data in systematic reviews. Using the most popular methods to handle missing binary outcome data, we investigated the implications of the accuracy of the extracted meta‐analysis on the random‐effects meta‐analysis results. Results Of 113 meta‐analyses from Cochrane reviews, 60 (53%) were judged as “unclearly” extracted (ie, no information on the outcome of completers but available information on how missing participants were handled) and 42 (37%) as “unacceptably” extracted (ie, no information on the outcome of completers as well as no information on how missing participants were handled). For the remaining meta‐analyses, it was judged that data were “acceptably” extracted (ie, information on the completers' outcome was provided for all trials). Overall, “unclear” extraction overestimated the magnitude of the summary odds ratio and the between‐study variance and additionally inflated the uncertainty of both meta‐analytical parameters. The only eligible Campbell review was judged as “unclear.” Conclusions Depending on the extent of missingness, the reporting quality of the systematic reviews can greatly affect the accuracy of the extracted meta‐analyses and by extent, the empirical performance of different methods to handle missingness.