Ensemble size: How suboptimal is less than infinity?

Ensemble forecasts are the method of choice in numerical weather prediction (NWP) to generate probabilistic forecasts. The number of members in an ensemble is an important factor in determining how well a probability distribution of a weather‐related variable can be estimated. Having only a finite number of members reduces the average skill such a probabilistic forecast can have. Increasing ensemble size is therefore desirable; however, ensemble size is also proportional to the computational cost. Having a small ensemble size limits the cost and makes other improvements, such as increases in spatial resolution, feasible. This article examines how average skill measures with metrics such as the continuous ranked probability score, the quantile score, and the Dawid–Sebastiani score converge with ensemble size. A numerical experiment with a 200 member ensemble using the European Centre for Medium‐Range Weather Forecasts (ECMWF) Integrated Forecasting System (IFS) model at a resolution of 29 km and a forecast range of 15 days provides data to compare the convergence of probabilistic skill in a current NWP system with theoretical expectations derived for perfectly reliable ensembles with exchangeable members. Results in the first part of the article can help users of operational NWP ensemble forecasts formulate their minimum requirement in terms of ensemble size. In the second part, requirements for scientists who test changes to NWP systems are examined. Using proper scores and fair scores, it is explored whether testing changes in the ensemble forecasts can be meaningful with fewer members than in the operational configuration. Results are based on medium‐range numerical experiments with 50 members. Two experiments test the activation of a representation of model uncertainty and three other experiments test changes in horizontal resolution from 29 to 18 km and from 29 to 45 km.