Premium
On the Comparison of Interval Forecasts
Author(s) -
Askanazi Ross,
Diebold Francis X.,
Schorfheide Frank,
Shin Minchul
Publication year - 2018
Publication title -
journal of time series analysis
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.576
H-Index - 54
eISSN - 1467-9892
pISSN - 0143-9782
DOI - 10.1111/jtsa.12426
Subject(s) - interval (graph theory) , quantile , mathematics , confidence interval , prediction interval , statistics , econometrics , calibration , interval estimation , combinatorics
We explore interval forecast comparison when the nominal confidence level is specified, but the quantiles on which intervals are based are not specified. It turns out that the problem is difficult, and perhaps unsolvable. We first consider a situation where intervals meet the Christoffersen conditions (in particular, where they are correctly calibrated), in which case the common prescription, which we rationalize and explore, is to prefer the interval of shortest length. We then allow for mis‐calibrated intervals, in which case there is a calibration‐length tradeoff. We propose two natural conditions that interval forecast loss functions should meet in such environments, and we show that a variety of popular approaches to interval forecast comparison fail them. Our negative results strengthen the case for abandoning interval forecasts in favor of density forecasts: Density forecasts not only provide richer information, but also can be readily compared using known proper scoring rules like the log predictive score, whereas interval forecasts cannot.