Open AccessSensitivity of Several Performance Measures to Displacement Error, Bias, and Event Frequency
Author(s) -
Michael E. Baldwin,
John S. Kain
Publication year - 2006
Publication title -
weather and forecasting
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.393
H-Index - 106
eISSN - 1520-0434
pISSN - 0882-8156
DOI - 10.1175/waf933.1
Subject(s) - event (particle physics) , statistic , sensitivity (control systems) , statistics , displacement (psychology) , measure (data warehouse) , computer science , econometrics , odds , mathematics , data mining , psychology , logistic regression , physics , quantum mechanics , electronic engineering , engineering , psychotherapist
The sensitivity of various accuracy measures to displacement error, bias, and event frequency is analyzed for a simple hypothetical forecasting situation. Each measure is found to be sensitive to displacement error and bias, but probability of detection and threat score do not change as a function of event frequency. On the other hand, equitable threat score, true skill statistic, and odds ratio skill score behaved differently with changing event frequency. A newly devised measure, here called the bias-adjusted threat score, does not change with varying event frequency and is relatively insensitive to bias. Numerous plots are presented to allow users of these accuracy measures to make quantitative estimates of sensitivities that are relevant to their particular application.