Promises and Problems of Systematic Reviews

Accessible online at: www.karger.com/journals/hed The number of clinical trials published in the area of cardiology is huge. Each year, over 1,000 trials of health care interventions within cardiology are identified for inclusion in the specialist trials register of the Cochrane Library (links via www.cochrane.org). Clearly, keeping up to date with new information is extremely difficult. With the advent of systematic reviews, where similar trials are grouped to provide a summary effect estimate, the accessibility of information for busy clinicians has improved. Whilst the potential benefits of this approach are many, the methodological quality of systematic reviews published in conventional journals and their relevance to clinical practice have been raised as potential problems. This is the subject of the report by Dr. Kleist [1] in this issue of Heart Drug. Cochrane systematic reviews differ from systematic reviews published in paper-based journals in several respects. The fundamental distinction is that authors are expected to update and amend them in light of relevant new data. There are also strict methodological guidelines to follow. Cochrane reviews are more likely to include a description of the inclusion and exclusion criteria, assess the methodological quality of individual trials and include non-English-language articles than systematic reviews published in paper-based journals [2]. These issues, particularly relating to the methodological quality of individual trials and potential sources of bias, have been highlighted in the report by Kleist [1]. Many of the problems associated with the interpretation and critical appraisal of systematic reviews are due to inadequate reporting. In an attempt to make interpretation more transparent, guidelines have recently been developed to improve the quality of reporting of systematic reviews (the QUOROM statement) [3]. Systematic reviews are certainly an advance compared to traditional ‘opinion’ reviews that cite evidence selectively to support a viewpoint. Another concern is the external validity (i.e. generalisability) of findings from systematic reviews, and hence their relevance to clinical practice. Clearly, the external validity of systematic reviews is dependent on the external validity of the trials included within them. There are two main concerns. First, patients who are recruited into clinical trials are not necessarily representative. There are often age restrictions and exclusions associated with comorbidity. Findings from such trials may not be applicable to the patient population as a whole. Secondly, trials may ask questions of little relevance to clinical practice. A study by the Cochrane Heart Group [4] examined the clinical relevance of 1,008 trials of health care interventions in cardiology published in 1998. The setting where the trials were conducted was reported in only 45% of studies, and of these, only 7% were community rather than hospital based. The majority of trials were of pharmacological interventions (61.5%). Most trials were of short duration and of relatively small size (recruiting less than 100 patients). Perhaps the most striking finding was that only half of the trials reported clinical events, and less than 4% reported quality of life as an outcome. Trials of