Using Length of Stay to Control for Unobserved Heterogeneity When Estimating Treatment Effect on Hospital Costs with Observational Data: Issues of Reliability, Robustness, and Usefulness

Objective To evaluate the sensitivity of treatment effect estimates when length of stay ( LOS ) is used to control for unobserved heterogeneity when estimating treatment effect on cost of hospital admission with observational data. Data Sources/Study Setting We used data from a prospective cohort study on the impact of palliative care consultation teams ( PCCT s) on direct cost of hospital care. Adult patients with an advanced cancer diagnosis admitted to five large medical and cancer centers in the United States between 2007 and 2011 were eligible for this study. Study Design Costs were modeled using generalized linear models with a gamma distribution and a log link. We compared variability in estimates of PCCT impact on hospitalization costs when LOS was used as a covariate, as a sample parameter, and as an outcome denominator. We used propensity scores to account for patient characteristics associated with both PCCT use and total direct hospitalization costs. Data Collection/Extraction Methods We analyzed data from hospital cost databases, medical records, and questionnaires. Our propensity score weighted sample included 969 patients who were discharged alive. Principal Findings In analyses of hospitalization costs, treatment effect estimates are highly sensitive to methods that control for LOS , complicating interpretation. Both the magnitude and significance of results varied widely with the method of controlling for LOS . When we incorporated intervention timing into our analyses, results were robust to LOS ‐controls. Conclusions Treatment effect estimates using LOS ‐controls are not only suboptimal in terms of reliability (given concerns over endogeneity and bias) and usefulness (given the need to validate the cost‐effectiveness of an intervention using overall resource use for a sample defined at baseline) but also in terms of robustness (results depend on the approach taken, and there is little evidence to guide this choice). To derive results that minimize endogeneity concerns and maximize external validity, investigators should match and analyze treatment and comparison arms on baseline factors only. Incorporating intervention timing may deliver results that are more reliable, more robust, and more useful than those derived using LOS ‐controls.