Premium
Exogenous factors matter when interpreting the results of an impact evaluation: a case study of rainfall and child health programme intervention in Rwanda
Author(s) -
Mukabutera Assumpta,
Thomson Dana R.,
HedtGauthier Bethany L.,
Atwood Sidney,
Basinga Paulin,
Nyirazinyoye Laetitia,
Savage Kevin P.,
Habimana Marcellin,
Murray Megan
Publication year - 2017
Publication title -
tropical medicine and international health
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.056
H-Index - 114
eISSN - 1365-3156
pISSN - 1360-2276
DOI - 10.1111/tmi.12995
Subject(s) - psychological intervention , wasting , intervention (counseling) , public health , developing country , environmental health , medicine , nursing , endocrinology , psychiatry , economics , economic growth
Objective Public health interventions are often implemented at large scale, and their evaluation seems to be difficult because they are usually multiple and their pathways to effect are complex and subject to modification by contextual factors. We assessed whether controlling for rainfall‐related variables altered estimates of the efficacy of a health programme in rural Rwanda and have a quantifiable effect on an intervention evaluation outcomes. Methods We conducted a retrospective quasi‐experimental study using previously collected cross‐sectional data from the 2005 and 2010 Rwanda Demographic and Health Surveys ( DHS ), 2010 DHS oversampled data, monthly rainfall data collected from meteorological stations over the same period, and modelled output of long‐term rainfall averages, soil moisture, and rain water run‐off. Difference‐in‐difference models were used. Results Rainfall factors confounded the PIH intervention impact evaluation. When we adjusted our estimates of programme effect by controlling for a variety of rainfall variables, several effectiveness estimates changed by 10% or more. The analyses that did not adjust for rainfall‐related variables underestimated the intervention effect on the prevalence of ARI by 14.3%, fever by 52.4% and stunting by 10.2%. Conversely, the unadjusted analysis overestimated the intervention's effect on diarrhoea by 56.5% and wasting by 80%. Conclusion Rainfall‐related patterns have a quantifiable effect on programme evaluation results and highlighted the importance and complexity of controlling for contextual factors in quasi‐experimental design evaluations.