Premium
Adjusting for information content when comparing forecast performance
Author(s) -
Andersson Michael K,
Aranki Ted,
Reslow André
Publication year - 2017
Publication title -
journal of forecasting
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.543
H-Index - 59
eISSN - 1099-131X
pISSN - 0277-6693
DOI - 10.1002/for.2470
Subject(s) - ranking (information retrieval) , computer science , monte carlo method , econometrics , statistics , artificial intelligence , mathematics
Cross‐institutional forecast evaluations may be severely distorted by the fact that forecasts are made at different points in time and therefore with different amounts of information. This paper proposes a method to account for these differences when analyzing an unbalanced panel of forecasts. The method computes the timing effect and the forecaster's ability simultaneously. Monte Carlo simulation demonstrates that evaluations that do not adjust for the differences in information content may be misleading. In addition, the method is applied to a real‐world dataset of 10 Swedish forecasters for the period 1999–2015. The results show that the ranking of the forecasters is affected by the proposed adjustment.