Open AccessA Note On the Maximum Peirce Skill Score
Author(s) -
Agostino Manzato
Publication year - 2007
Publication title -
weather and forecasting
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.393
H-Index - 106
eISSN - 1520-0434
pISSN - 0882-8156
DOI - 10.1175/waf1041.1
Subject(s) - contingency table , statistics , event (particle physics) , mathematics , threshold limit value , econometrics , binary number , point (geometry) , value (mathematics) , computer science , arithmetic , medicine , physics , geometry , quantum mechanics , environmental health
Binary classifiers are obtained from a continuous predictor using a threshold to dichotomize the predictor value into event occurrence and nonoccurrence classes. A contingency table is associated with each threshold, and from this table many statistical indices (like skill scores) can be computed. This work shows that the threshold that maximizes one of these indices [the Peirce skill score (PSS)] has some important properties. In particular, at that threshold the ratio of the two likelihood distributions is always 1 and the event posterior probability is equal to the event prior probability. These properties, together with the consideration that the maximum PSS is the point with the “most skill” on the relative operating characteristic curve and the point that maximizes the forecast value, suggest the use of the maximum PSS as a good scalar measure of the classifier skill. To show that this most skilled point is not always the best one for all the users, a simple economic cost model is presented.