Premium
A comparison of methods for health policy evaluation with controlled pre‐post designs
Author(s) -
O'Neill Stephen,
Kreif Noemi,
Sutton Matt,
Grieve Richard
Publication year - 2020
Publication title -
health services research
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.706
H-Index - 121
eISSN - 1475-6773
pISSN - 0017-9124
DOI - 10.1111/1475-6773.13274
Subject(s) - confidence interval , payment , health care , medicine , control (management) , point estimation , hip fracture , point (geometry) , actuarial science , statistics , computer science , economics , mathematics , osteoporosis , artificial intelligence , endocrinology , world wide web , economic growth , geometry
Objective To compare interactive fixed effects (IFE) and generalized synthetic control (GSC) methods to methods prevalent in health policy evaluation and re‐evaluate the impact of the hip fracture best practice tariffs introduced for hospitals in England in 2010. Data Sources Simulations and Hospital Episode Statistics. Study Design Best practice tariffs aimed to incentivize providers to deliver care in line with guidelines. Under the scheme, 62 providers received an additional payment for each hip fracture admission, while 49 providers did not. We estimate the impact using difference‐in‐differences (DiD), synthetic control (SC), IFE, and GSC methods. We contrast the estimation methods' performance in a Monte Carlo simulation study. Principal Findings Unlike DiD, SC, and IFE methods, the GSC method provided reliable estimates across a range of simulation scenarios and was preferred for this case study. The introduction of best practice tariffs led to a 5.9 (confidence interval: 2.0 to 9.9) percentage point increase in the proportion of patients having surgery within 48 hours and a statistically insignificant 0.6 (confidence interval: −1.4 to 0.4) percentage point reduction in 30‐day mortality. Conclusions The GSC approach is an attractive method for health policy evaluation. We cannot be confident that best practice tariffs were effective.