Estimating treatment effect via simple cross design synthesis

Randomized controlled trials (RCTs) are the traditional gold standard evidence for medical decision‐making. However, protocols that limit enrollment eligibility introduce selection error that severely limits a RCT's applicability to a wide range of patients. Conversely, high quality observational data can be representative of entire populations, but freedom to choose treatment can bias estimators based on this data. Cross design synthesis (CDS) is an approach to combining both RCT and observational data in a single analysis that capitalizes on the RCT's strong internal validity and the observational study's strong external validity. We proposed and assessed a simple estimator of effect size based on the CDS approach. We evaluated its properties within a formal framework of causal estimation and compared our estimator with more traditional estimators based on single sources of evidence. We show that under ideal conditions the simple CDS estimator is unbiased whenever the observational data‐based estimators' treatment selection error is constant across those who are and are not eligible for RCT participation. Whereas this assumption may not often hold in practice, assumptions required for the unbiasedness of usual single‐source estimators may also be implausible. We show that, under some reasonable data assumptions, our simple CDS estimator has smaller bias and better coverage than commonly used estimates based on randomized or observational studies alone. Copyright © 2011 John Wiley & Sons, Ltd.