Making sense of meta‐analysis

Abstract Meta‐analyses of clinical trials have become increasingly popular over the last decade. This technique pools data from different studies in order to provide an overview and obtain more precise estimates of treatment effects. Meta‐analyses may thus detect or exclude small but clinically relevant effects or side effects. A number of criteria have to be met for a reliable meta‐analysis of clinical trials. Only randomized trials analysed according to the intention‐to‐treat principle with complete follow‐up information and comparable, objective outcome assessments should generally be included. Treatments must be comparable from a clinical and pharmacological point of view and similarity of lengths of follow‐up is important when the effect of a treatment varies over time. In this review, the statistical models commonly used are discussed and an example of a well conducted meta‐analysis is given. Several unresolved problems pertinent to meta‐analysis are considered. It is difficult to ensure that all relevant trials have been identified. Selective inclusion of trials may distort results. Important differences in the quality of the studies are generally not taken into account in meta‐analyses. Statistical testing for heterogeneity and the use of random effects models cannot resolve the problems which arise when study results are highly variable. Finally, the clinical application of the results from meta‐analysis may be problematic, since the patients included in the constituent trials are often not representative of the range of patients seen in clinical practice. Meta‐analyses should be viewed as a powerful addition to traditional methods of reviewing medical evidence. In interpreting their results, however, care should be taken not to go beyond the available data.