Appraisal of Evidence and Control of Bias in the Kidney Disease Outcomes Quality Initiative Guideline Development Process

It is not evidence alone but a combination of evidence and judgment that drives most medical decision making. Limitations afflict even high-quality evidence; subjectivity influences even the most objective experts. Therefore, in the absence of successful clinical practice guidelines, prevalent practice suffers from an overreliance on flawed information and a susceptibility to uncontrolled bias. Ascribing causality to observational results (1), assuming the equivalence of surrogate and direct patient outcomes (2), and pursuing practices or interventions despite a dearth of evidence for safety (3) are signs that clinical practice may be operating without critical appraisal of the evidence or sufficient restraint of subjectivity. Practice under such conditions provides fertile ground for the seeds of influence, whatever their source.To be successful, a clinical practice guideline development process must deal effectively with both the limitations of available evidence and the inevitability of expert subjectivity. The required principles are straightforward: Tease out the strengths and the limitations of evidence, and control potential bias by adhering to scientific and methodologic rigor; foster an interdisciplinary and independent working group; and use an open, structured, and systematic development process. These were the founding principles for the National Kidney Foundation Kidney Disease Outcomes Quality Initiative (NKF-KDOQI) (4,5); they have helped to make KDOQI the pride of the nephrology community and the envy of those who develop non-nephrology guidelines, and they served as the organizational framework for formulating the 2006 KDOQI Clinical Practice Guidelines and Clinical Practice Recommendations for Anemia in Chronic Kidney Disease (6).Adhering to these principles serves as the strongest available antidote against potentially negative influence on clinical practice in our community, whether the source of the influence is industry, incomplete information, or unexamined belief. Adhering to scientific and methodologic rigor, for example, requires recognition that the source of the evidence affects …