A diagnostic suite to assess NWP performance

A suite of numerical weather prediction (NWP) verification diagnostics applicable to both scalar and vector variables is developed, highlighting the normalization and successive decomposition of model errors. The normalized root‐mean square error (NRMSE) is broken down into contributions from the normalized bias (NBias) and the normalized pattern error (NPE). The square of NPE, or the normalized error variance α , is further analyzed into phase and amplitude errors, measured respectively by the correlation and the variance similarity. The variance similarity diagnostic is introduced to verify variability e.g. under different climates. While centered RMSE can be reduced by under‐prediction of variability in the model, α penalizes over‐ and under‐prediction of variability equally. The error decomposition diagram, the correlation‐similarity diagram and the anisotropy diagram are introduced. The correlation‐similarity diagram was compared with the Taylor diagram: it has the advantage of analyzing the normalized error variance geometrically into contributions from the correlation and variance similarity. Normalization of the error metrics removes the dependence on the inherent variability of a variable and allows comparison among quantities of different physical units and from different regions and seasons. This method was used to assess the Coupled Ocean/Atmospheric Mesoscale Prediction System (COAMPS). The NWP performance degrades progressively from the midlatitudes through the sub‐tropics to the tropics. But similar cold and moist biases are noted and position and timing errors are the main cause of pattern errors. Although the suite of metrics is applied to NWP verification here, it is generally applicable as diagnostics for differences between two data sets.