Premium
Forecasting Residential Water Consumption in California: Rethinking Model Selection
Author(s) -
Buck Steven,
Auffhammer Maximilian,
Soldati Hilary,
Sunding David
Publication year - 2020
Publication title -
water resources research
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.863
H-Index - 217
eISSN - 1944-7973
pISSN - 0043-1397
DOI - 10.1029/2018wr023965
Subject(s) - sample (material) , predictive modelling , consumption (sociology) , ensemble forecasting , selection (genetic algorithm) , term (time) , model selection , sample size determination , econometrics , computer science , investment (military) , range (aeronautics) , statistics , mathematics , machine learning , engineering , social science , chemistry , physics , chromatography , quantum mechanics , sociology , politics , political science , law , aerospace engineering
Urban water managers use forecasts of water consumption to determine management decisions and investment choices. Public reports show that water utilities rely on forecast models that are not selected based on their out‐of‐sample prediction performance; further, these reports frequently only present a single forecast instead of a range of forecasts. In our review of the academic literature on forecasting long‐term water consumption, only a few analyses consider out‐of‐sample prediction performance measures to assess prediction ability. In none of these analyses did out‐of‐sample prediction performance drive model selection. Ensemble‐type long‐term forecasts based on multiple models were also lacking. Using annual data on single‐family residential water consumption in Southern California, we show that predictive ability varies drastically depending on how the forecast model is selected. As an illustration of how forecast performance is affected when the criteria for model selection does support forecasting objectives, we compare statistical models with the best in‐sample and out‐of‐sample prediction performance. We find that the models with the best in‐sample performance over‐estimate consumption five years out by 10%‐25% compared to actual consumption. In contrast, the top 1% of models selected based on out‐of‐sample prediction performance came within 1% of actual consumption. Finally, we compare the performance of our ensemble‐type forecasts to those reported in public documents derived from models selected based on non‐out‐of‐sample prediction performance criteria. Our results highlight the benefits of (i) using an out‐of‐sample evaluation criterion to guide model selection and (ii) reporting ensemble forecasts in lieu of a single forecast.