Open AccessDecomposition of a New Proper Score for Verification of Ensemble Forecasts
Author(s) -
Hilary Christensen
Publication year - 2015
Publication title -
monthly weather review
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.862
H-Index - 179
eISSN - 1520-0493
pISSN - 0027-0644
DOI - 10.1175/mwr-d-14-00150.1
Subject(s) - brier score , skewness , forecast verification , standard deviation , predictability , forecast skill , quantitative precipitation forecast , probabilistic logic , ensemble forecasting , consensus forecast , probabilistic forecasting , scoring rule , term (time) , range (aeronautics) , reliability (semiconductor) , econometrics , meteorology , computer science , statistics , mathematics , precipitation , artificial intelligence , power (physics) , physics , quantum mechanics , materials science , composite material
A new proper score, the error-spread score (ES), has recently been proposed for evaluation of ensemble forecasts of continuous variables. The ES is formulated with respect to the moments of the ensemble forecast. It is particularly sensitive to evaluating how well an ensemble forecast represents uncertainty: is the probabilistic forecast well calibrated? In this paper, it is shown that the ES can be decomposed into its reliability, resolution, and uncertainty components in a similar way to the Brier score. The first term evaluates the reliability of the forecast standard deviation and skewness, rewarding systems where the forecast moments reliably indicate the properties of the verification. The second term evaluates the resolution of the forecast standard deviation and skewness, and rewards systems where the forecast moments vary from the climatological moments according to the predictability of the atmospheric flow. The uncertainty term depends only on the observed error distribution and is independent of the forecast standard deviation or skewness. The decomposition was demonstrated using forecasts made with the European Centre for Medium-Range Weather Forecasts ensemble prediction system, and was able to identify the source of the skill in the forecasts at different latitudes.