Evaluating Systematic Reviews and Meta-Analyses

Systematic review and meta-analysis procedures make use of explicit methods to methodically search and critically appraise and synthesize the medical care research literature. The methods involve refining a clinical question, designing a search procedure to find eligible studies, and determining the validity of the eligible studies. Independent data extraction by two or more reviewers is preferred. Agreement between the reviewers with respect to relevance and validity should be measured. Meta-analysis procedures estimate an overall average effect from the individual study effects and determine whether these effects appear to measure the same relationship (that is, the studies are not heterogeneous). In the inverse variance method, which is most frequently applied, the overall effect is a weighted average of the individual study effects, where each weight is the inverse of the study variance. To evaluate a systematic review, first determine whether it addresses a question that is relevant to the patients, treatments, and outcomes that are usual in your clinical practice. Then assess the validity of the systematic review, which is reflected by quality of the individual studies, the rigor with which the systematic methods were applied, and the extent of heterogeneity. If the results of the systematic review are valid, then is the effect important enough to make a difference in your clinical practice? Applying the results to an individual patient involves the absolute treatment effect or the number needed to treat, and an awareness of the patient's specific level of risk and personal preferences.