Forecasting Conditional Probabilities of Binary Outcomes under Misspecification

We consider constructing probability forecasts from a parametric binary choice model under a large family of loss functions (“scoring rules”). Scoring rules are weighted averages over the utilities that heterogeneous decision makers derive from a publicly announced forecast (Schervish, 1989). Using analytical and numerical examples, we illustrate howdifferent scoring rules yield asymptotically identical results if the model is correctly specified. Under misspecification, the choice of scoring rule may be inconsequential under restrictive symmetry conditions on the data-generating process. If these conditions are violated, typically the choice of a scoring rule favors some decision makers over others.