Open AccessA skill score based on economic value for probability forecasts
Author(s) -
Wilks D S
Publication year - 2001
Publication title -
meteorological applications
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.672
H-Index - 59
eISSN - 1469-8080
pISSN - 1350-4827
DOI - 10.1017/s1350482701002092
Subject(s) - econometrics , value (mathematics) , range (aeronautics) , statistics , function (biology) , receiver operating characteristic , conditional probability , mathematics , computer science , materials science , evolutionary biology , composite material , biology
An approach to evaluating probability forecasts for dichotomous events, based on their economic value over all possible cost/loss ratio decision problems, is proposed. The resulting Value Score (VS) curve shows non‐dimensionalised relative economic value as a function of the cost/loss ratios for different decision‐makers, over their full meaningful range. The VS curve is similar in terms of computational mechanics and graphical display to the Relative Operating Characteristic (ROC) curve, but the ROC curve is shown to be insensitive to either conditional or unconditional biases and thus to reflect potential rather than actual skill. The possibility of collapsing the VS curve into a single scalar score is addressed, and it is shown that the results can depend very strongly on the assumed distribution of cost/loss ratios in the community of forecast users. Copyright © 2001 Royal Meteorological Society