Program evaluation within the research translation framework

A program evaluation is the systematic collection and analysis of data designed to examine the value of a program in terms of implementation, efficiency and effectiveness. In healthcare, a program evaluation can demonstrate improved patient and economic outcomes or, if an improvement is not shown, the evaluation can potentially minimise avoidable waste for the health service by reducing the provision of low-value care practices. The evaluation may also reduce avoidable waste in research by improving the rate of successful replication of basic research and translation to healthcare. Program evaluation provides a bridge between research and clinical practice. The purpose of this editorial is to highlight that program evaluation lies within the research translation framework, as well as to discuss the key elements of program evaluation methodology and give examples for appropriate application of a program evaluation. Program evaluation is a part of the research translation framework in healthcare, similar to many quality improvement activities that aim to test implementation of research in a local setting before widespread uptake. Program evaluation can be considered as a knowledge transfer step within the phases of research translation; it is reflected in the third stage of research translation, known as T3 (from guidelines to health practice), and in the fourth stage of research translation, known as T4 (from health practice to population health outcomes). Program evaluation includes both knowledge transfer (closing the ‘know-do’ gap between what is known and what is done) and implementation science (‘how to’ implement the knowledge). Program evaluation has three distinct components: formative, process and summative evaluations. The formative phase is designed to help shape the evaluation by assessing the program design and piloting it prior to implementation across the health service. For example, the formative evaluation may include: a gap analysis to report on current state compared to evidence-based clinical guidelines; a needs assessment to report the perceived want or need from those influencing or impacted by the program; and an analysis of barriers and facilitators to inform the implementation strategy. Process evaluation is designed to determine the extent to which a programwas implemented according to plan. Should a program fail, it is imperative to distinguish between a defective program and a defective implementation strategy. For example, the process evaluation outcome may simply include measured adherence to a published implementation framework using a single metric such as that for stroke guidelines or more comprehensively consider success/lack of success to achieve the eight recommended implementation outcomes, including: acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability. The summative evaluation may be more familiar to traditional researchers and clinicians, as this is an assessment of the overall impact of the program based on the program objectives and includes analysis of the health and economic outcomes, both intended and