When the best laid plans go astray: a case study in pragmatic approaches to evaluation

Many researchers recognise the importance of conducting evaluations in accordance with clearly articulated plans and frameworks which are developed at the outset of the project and underpin all of the subsequently undertaken research activities. However, for reasons often beyond the control of the evaluators, research methods and data collection instruments identified in these plans and frameworks may become unfeasible or inappropriate at different stages of the evaluation period. Mitigating the potential impact of these events and circumstances can be incredibly challenging for even the most seasoned researcher, who at the end of the day is still required to answer key evaluation questions, one of which may be ‘Did it work?’ Using the evaluation of the Family Group Conferencing pilot project (NSW) for illustrative purposes, this paper highlights the benefits of reflexive, adaptive and pragmatic approaches to evaluation and of involving project stakeholders in the development of evaluation designs and research methods.