Enhancing Heavy Rain Nowcasting with Multimodal Data: Integrating Radar and Satellite Observations

TL;DR

This work tackles the challenge of nowcasting heavy rainfall in urban settings by fusing radar and satellite observations. It introduces two U‑Net–based models (radar-only and radar+satellite) and demonstrates that multimodal fusion improves forecast accuracy at lead times of 5, 15, and 30 minutes, with notable gains in extreme rainfall detection. Evaluations on NRW summer data and a 2021 flood case show consistent CSI and FSS improvements for the multimodal model, particularly at short lead times, enabling more reliable early warnings. The study highlights the practical value of satellite-derived atmospheric context in augmenting radar measurements and points to future work on broader regions and advanced architectures to further boost nowcasting performance.

Abstract

The increasing frequency of heavy rainfall events, which are a major cause of urban flooding, underscores the urgent need for accurate precipitation forecasting - particularly in urban areas where localized events often go undetected by ground-based sensors. In Germany, only 17.3% of hourly heavy rain events between 2001 and 2018 were recorded by rain gauges, highlighting the limitations of traditional monitoring systems. Radar data are another source that effectively tracks ongoing precipitation; however, forecasting the development of heavy rain using radar alone remains challenging due to the brief and unpredictable nature of such events. Our focus is on evaluating the effectiveness of fusing satellite and radar data for nowcasting. We develop a multimodal nowcasting model that combines both radar and satellite imagery for predicting precipitation at lead times of 5, 15, and 30 minutes. We demonstrate that this multimodal strategy significantly outperforms radar-only approaches. Experimental results show that integrating satellite data improves prediction accuracy, particularly for intense precipitation. The proposed model increases the Critical Success Index for heavy rain by 4% and for violent rain by 3% at a 5-minute lead time. Moreover, it maintains higher predictive skill at longer lead times, where radar-only performance declines. A qualitative analysis of the severe flooding event in the state of North Rhine-Westphalia, Germany in 2021 further illustrates the superior performance of the multimodal model. Unlike the radar-only model, which captures general precipitation patterns, the multimodal model yields more detailed and accurate forecasts for regions affected by heavy rain. This improved precision enables timely, reliable, life-saving warnings. Implementation available at https://github.com/RamaKassoumeh/Multimodal_heavy_rain

Figures (4)

• Figure 1: The figure shows the ground truth of a flooding event in the state of North Rhine-Westphalia, Germany, on July 14, 2021, along with predictions from two models: Radar-only and Multimodal (Radar+Satellite) at a 30-minute lead time. Orange and red areas indicate heavy rainfall, with red representing higher precipitation intensities. This event led to catastrophic flooding in the region. The Multimodal model outperforms the Radar-only model, particularly in regions of intense rainfall where the Radar-only model fails to capture the full magnitude of the event. These results demonstrate that satellite data is crucial for accurate prediction of heavy rain scenarios. Capturing the correct magnitude of such events is essential for issuing timely warnings and enabling an effective first response, especially in light of similar extreme weather events that have recently occurred across Europe.
• Figure 2: Visualization of the 11 satellite bands and the radar image.
• Figure 3: Architecture of the multimodal model. The model (for 5 min lead time) takes as input a sequence of six time steps (from $t{-}30$ to $t{-}5$), where each frame includes 12 channels: 1 radar image and 11 satellite bands. These inputs are processed by a 3D U-Net architecture consisting of 3D convolutional layers, max-pooling, and upsampling operations. The model outputs a single radar image predicting precipitation at $t$. Skip connections and concatenation operations are used between encoder and decoder stages to preserve spatial and temporal information.
• Figure 4: This figure shows the ground truth radar image (target) of a flooding event in North Rhine-Westphalia on July 14, 2021, along with the predictions at time $t$ from two models: a Radar-only model and a Multimodal model. Both models use the same input sequence of radar images from $t{-}55$ to $t{-}30$ minutes. Additionally, the Multimodal model uses satellite images from the same time period. The color bar on the right indicates precipitation intensity in mm/h. The Multimodal model clearly outperforms the Radar-only model, particularly in regions of heavy rainfall, where the Radar-only model underestimates the intensity of the event.