Double Sampling with Multiple Imputation to Answer Large Sample Meta‐Research Questions: Introduction and Illustration by Evaluating Adherence to Two Simple CONSORT Guidelines

BACKGROUND High throughput meta‐research methods have made large meta‐research questions feasible, possibly at the cost of accuracy. We predict that double sampling with multiple imputation (DS+MI) will result in a precise and accurate estimate of adherence to CONSORT title and abstract guidelines. METHODS: We retrieved all PubMed entries with filters: RCT; human; abstract available; and English (n=322,107). We evaluated these with a lower rigor, higher throughput (R LO T HI ) search heuristics method. We evaluated a subsample of 500 entries and evaluated them using a higher rigor, lower throughput (R HI T LO ) human rating method. R HI T LO evaluations for the large sample were estimated with multiple imputation. RESULTS: The R HI T LO and R LO T HI methods in the subsample agreed (phi coefficients: title=1.00, abstract=0.92). Compliance has increased over time, with non‐US countries improving more rapidly. DS+MI logistic regression estimates were more precise than subsample estimates (e.g., 95% CI for change in title and abstract compliance by Year: subsample R HI T LO 1.050‐1.174 vs. DS+MI 1.082‐1.151). As evidence of improved accuracy, DS+MI coefficient estimates were closer to R HI T LO than the large sample R LO T HI . CONCLUSIONS Our results support our hypothesis that DS+MI improves precision and accuracy. This method is flexible and may provide a practical way to examine large corpora of literature. Support: NIH grants K12GM088010, P30DK056336, T32HL072757, and R25HL124208.