A Technique for Verification of Convection-Permitting NWP Model Deterministic Forecasts of Lightning Activity

This manuscript introduces a new technique for evaluating lightning forecasts from convection-permitting models. In recent years, numerical weather prediction models at the convection-permitting scales (horizontal grid resolutions of 1–5 km) have been able to produce realistic-looking forecasts of lightning activity when compared with observations. However, it is challenging to assess what value these forecasts add above standard large-scale indices. Examining this problem, it is found that existing skill scores and neighborhood verification methods are unable to cope with both the double-penalty effect and the model’s variable frequency bias. A displacement distance and a quasi-symmetric distance score are introduced based on the distance between the model and the observations, the latter showing any improvement the forecast has over a completely “hedged” forecast. This can be combined with a domain-improved contingency table and comparisons between modeled and observed lightning flashes to evaluate the forecast performance in three important dimensions: coverage, distance, and intensity. The verification metric is illustrated with a single case, which shows that the convective-scale U.K. variable resolution model (UKV) delivers improved forecasts compared with the large-scale indices in both coverage and distance. Additionally, a month-long analysis is performed, which reveals that the coverage of lightning is in good agreement with the observations; lightning is displaced by the model by a distance on the order of 50–75 km, but the model overpredicts the lightning intensity by at least a factor of 6 after observational detection efficiencies have been considered.