Improving Medium Range Severe Weather Prediction through Transformer Post-processing of AI Weather Forecasts

TL;DR

This study tackles the challenge of medium-range severe weather prediction by combining AI NWP forecasts from Pangu-Weather with a decoder-only transformer post-processing framework that treats lead times as sequential tokens. The DOT architecture learns temporal dependencies across forecast days to generate probabilistic convective hazard guidance, outperforming a dense neural network baseline and traditional GFS-based post-processing, especially for Days 3–6. AI-driven forecasts, particularly Pangu-Weather initialized with ERA5-like analyses, demonstrate superior skill relative to GFS, and ensemble averaging across multiple inputs yields the strongest performance with improved reliability. Explainable AI via Integrated Gradients provides interpretable, day-scale feature attributions, supporting understanding of how input variables drive hazard predictions and highlighting the potential for operational deployment with improved accuracy and interpretability.

Abstract

Improving the skill of medium-range (3-8 day) severe weather prediction is crucial for mitigating societal impacts. This study introduces a novel approach leveraging decoder-only transformer networks to post-process AI-based weather forecasts, specifically from the Pangu-Weather model, for improved severe weather guidance. Unlike traditional post-processing methods that use a dense neural network to predict the probability of severe weather using discrete forecast samples, our method treats forecast lead times as sequential ``tokens'', enabling the transformer to learn complex temporal relationships within the evolving atmospheric state. We compare this approach against post-processing of the Global Forecast System (GFS) using both a traditional dense neural network and our transformer, as well as configurations that exclude convective parameters to fairly evaluate the impact of using the Pangu-Weather AI model. Results demonstrate that the transformer-based post-processing significantly enhances forecast skill compared to dense neural networks. Furthermore, AI-driven forecasts, particularly Pangu-Weather initialized from high resolution analysis, exhibit superior performance to GFS in the medium-range, even without explicit convective parameters. Our approach offers improved accuracy, and reliability, which also provides interpretability through feature attribution analysis, advancing medium-range severe weather prediction capabilities.

Figures (14)

• Figure 1: Comparison of post-processing frameworks: (a) The Dense Neural Network (DNN) framework takes a discrete sample with an input shape of (Batch, Feature) and outputs a severe weather probability for a single day, with an output shape of (Batch, 1). Shapes above arrows indicate the output shape of each layer. (b) The Decoder-only Transformer (DOT) framework utilizes a consecutive sample of weather forecasts from Day 1 to Day 8, with an input shape of (Batch, Sequence, Feature), and outputs severe weather probabilities for each day from Day 1 to Day 8, resulting in an output shape of (Batch, Sequence, 1). Description of hyperparameters for the PyTorch paszke2019pytorch TransformerDecoder ("d_model": embedding/layer dimension, "num_layers": number of stacked layers, "nhead": number of attention heads, "tgt_mask=True": enables causal masking) are further elaborate in Section \ref{['sec: post-processing framework']}.
• Figure 2: Number of days (12 UTC -- 12 UTC) with $\geq$ 1 storm report occurring within each 80-km grid box during the verification period (1 February 2024 -- 22 July 2024). Report day totals were smoothed using a Gaussian smoother with $\sigma$ = 80 km.
• Figure 3: The plots show (a) Base Rate BSS, (b) PRAUC, and (c) ROCAUC performance comparison as a function of forecast day between DOT and DNN architectures using GFS forecast. The lines in the bottom right legend represent different models or configuration. Statistical significance at the 95% confidence level was assessed for the specific model pairs listed in the legend below the plot. For each pair, 1,000 bootstrap resamples of the skill difference were computed at each lead time. A symbol is shown if the corresponding bootstrapped 95% confidence interval lies entirely above zero, indicating a statistically significant improvement.
• Figure 4: Similar to Fig. \ref{['fig:dot_vs_dnn_gfs']}, this figure compares the performance of the DOT post-processing architecture, but using Pangu-Weather forecasts versus GFS forecasts as input.
• Figure 5: Precision-Recall curves for different forecast models (DOT(GFS), DOT(Pangu-ERA5), DOT(Pangu-GFS), DOT(Pangu-HRES)) across lead times from Day 3 (D3) to Day 6 (D6). Each panel displays the performance at a fixed probability threshold: (a) 0.15, (b) 0.30, (c) 0.45, and (d) 0.60. Precision (1 - False Alarm Ratio) is plotted against the Recall (True Positive Rate) on a logarithmic scale.