Premium
Double Sampling with Multiple Imputation to Answer Large Sample Meta‐Research Questions: Introduction and Illustration by Evaluating Adherence to Two Simple CONSORT Guidelines
Author(s) -
Capers Patrice,
Brown Andrew,
Dawson John,
Allison David
Publication year - 2015
Publication title -
the faseb journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.709
H-Index - 277
eISSN - 1530-6860
pISSN - 0892-6638
DOI - 10.1096/fasebj.29.1_supplement.735.1
Subject(s) - imputation (statistics) , statistics , computer science , logistic regression , mathematics , medicine , missing data
BACKGROUND High throughput meta‐research methods have made large meta‐research questions feasible, possibly at the cost of accuracy. We predict that double sampling with multiple imputation (DS+MI) will result in a precise and accurate estimate of adherence to CONSORT title and abstract guidelines. METHODS: We retrieved all PubMed entries with filters: RCT; human; abstract available; and English (n=322,107). We evaluated these with a lower rigor, higher throughput (R LO T HI ) search heuristics method. We evaluated a subsample of 500 entries and evaluated them using a higher rigor, lower throughput (R HI T LO ) human rating method. R HI T LO evaluations for the large sample were estimated with multiple imputation. RESULTS: The R HI T LO and R LO T HI methods in the subsample agreed (phi coefficients: title=1.00, abstract=0.92). Compliance has increased over time, with non‐US countries improving more rapidly. DS+MI logistic regression estimates were more precise than subsample estimates (e.g., 95% CI for change in title and abstract compliance by Year: subsample R HI T LO 1.050‐1.174 vs. DS+MI 1.082‐1.151). As evidence of improved accuracy, DS+MI coefficient estimates were closer to R HI T LO than the large sample R LO T HI . CONCLUSIONS Our results support our hypothesis that DS+MI improves precision and accuracy. This method is flexible and may provide a practical way to examine large corpora of literature. Support: NIH grants K12GM088010, P30DK056336, T32HL072757, and R25HL124208.