MAG-Net: Physics-Aware Multi-Modal Fusion of Geostationary Satellite and Radar for Severe Convective Precipitation Nowcasting

Abstract

Radar-based convective precipitation nowcasting suffers from rapid performance degradation beyond 30 minutes due to missing thermodynamic variables. Existing deep learning models also face blurring effects, training instability, and limited interpretability. To address this, we propose MAG-Net, a Physics-Aware Multi-modal Attention-guided Generator Network. It integrates radar dynamics with selected geostationary satellite channels (IR 10.8, WV 7.1, BTD) to incorporate thermodynamic and microphysical precursors. MAG-Net features a Dual-Stream Encoder for heterogeneous modalities and a Symmetric Dual-Head Decoder optimizing reflectivity regression and event probability via an uncertainty-weighted multi-task strategy. Furthermore, an inference-time Gradient-Preserving Fusion (GPF) strategy combines probabilistic constraints with regression details for better high-frequency texture retention. Experiments on a large-scale dataset (2018-2023) over southeastern China show MAG-Net outperforms deterministic (e.g., CPrecNet) and generative (e.g., DGMR) baselines. Specifically, it improves CSI40 by 0.083 (0.172 to 0.255) over CPrecNet, enhancing intense convective echo detection. Finally, Integrated Gradients (IG) analysis reveals the model's reliance on satellite inputs increases with forecast lead time and convective intensity, confirming that satellite data captures critical precursors for severe weather prediction.

Figures (9)

• Figure 1: Schematic overview of the proposed MAG-Net (Multi-modal Attention-guided Generator Network). (a) The architecture features a symmetric dual-head design that simultaneously predicts pixel-wise intensity (Regression Head) and probability maps (Classification Head). The Multi-modal Fusion Module integrates spatiotemporal features from radar sequences and satellite channels (WV 7.1 $\mu$m, IR 10.8 $\mu$m, and BTD 10.8–12.0 $\mu$m). (b) The Gradient-Preserving Fusion Strategy combines the low-frequency components from the regression output with the high-frequency details refined by the classification probability map. Note that the classification task serves as a uncertainty-weighted multi-task learning strategy to guide the regression task toward structurally coherent predictions, particularly for high-intensity echoes.
• Figure 2: Quantitative performance comparison on the test set. (a)–(c) Critical Success Index (CSI) at thresholds of 20, 30, and 40 dBZ across forecast lead times. (d)–(f) Fractions Skill Score (FSS) at different neighborhood scales. (g) Performance Diagram at the 90-minute lead time. Note: In panel (g), the 12 dBZ threshold is plotted to represent the boundary between precipitation and non-precipitation, as the data normalization lower bound is set to 10 dBZ. MAG-Net (red stars) shows a favorable trade-off between Probability of Detection (POD) and Success Ratio (1 - FAR), particularly at higher intensity thresholds (30 and 40 dBZ).
• Figure 3: Overall performance evaluation. (a) Mean Absolute Error (MAE) and (b) Root Mean Square Error (RMSE) averaged over all lead times (lower is better). (c) Temporal evolution of RMSE over the 90-minute forecast horizon. Comparison setup: The SOTA group includes representative single-modal deterministic baselines. The MM-Models group compares the proposed MAG-Net against MM-Reg, an architectural variant trained with a pure regression objective (excluding the classification head). The results suggest that the dual-head constraint helps mitigate error accumulation relative to the pure regression variant and radar-only baselines.
• Figure 4: Spectral consistency analysis. (a) Temporal evolution of the Band Power Ratio (BPR), defined as $\mathrm{BPR}(t)=\frac{\sum_{k=8}^{40} P_{\mathrm{pred}}(k,t)}{\sum_{k=8}^{40} P_{\mathrm{gt}}(k,t)}$, where $P(k,t)$ denotes the radially averaged power at wavenumber $k$ for the $t$-th lead time. DGMR (green dashed line) shows relatively higher band power at early lead times but degrades over time. MAG-Net (red solid line) maintains higher band power at later lead times, indicating improved retention of high-frequency energy. (b) Radially averaged Power Spectral Density (PSD) at the 90-minute lead time. The zoom-in window highlights the high-frequency tail. MAG-Net shows a closer alignment to the Ground Truth (black line) in the highlighted band compared to regression baselines, while DGMR exhibits larger spectral decay at 90 minutes under this setting.
• Figure 5: Qualitative visualization of a representative convective initiation event on June 6, 2023, at 18:50 BJT. Rows display the Ground Truth and predictions from key deterministic baselines (CPrecNet, SimVPv2) compared to the proposed multi-modal variants (MM-Reg, MM-Dual). For visual clarity, only deterministic baselines are shown. While single-modal models may struggle to capture incipient echo formation, MAG-Net (bottom row) better captures the emergence and intensification of the convective core, consistent with the use of satellite precursors.