Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FusionCast: Enhancing Precipitation Nowcasting with Asymmetric Cross-Modal Fusion and Future Radar Priors

Henan Wang, Shengwu Xiong, Yifang Zhang, Wenjie Yin, Chen Zhou, Yuqiang Zhang, Pengfei Duan

Abstract

Deep learning has significantly improved the accuracy of precipitation nowcasting. However, most existing multimodal models typically use simple channel concatenation or interpolation methods for data fusion, which often overlook the feature differences between different modalities. This paper therefore proposes a novel precipitation nowcasting optimisation framework called FusionCast. This framework incorporates three types of data: historical precipitable water vapour (PWV) data derived from global navigation satellite system (GNSS) inversions, historical radar based quantitative precipitation estimation (QPE), and forecasted radar QPE serving as a future prior. The FusionCast model comprises two core modules: the future prior radar QPE processing Module, which forecasts future radar data; and the Radar PWV Fusion (RPF) module, which uses a gate mechanism to efficiently combine features from various sources. Experimental results show that FusionCast significantly improves nowcasting performance.

FusionCast: Enhancing Precipitation Nowcasting with Asymmetric Cross-Modal Fusion and Future Radar Priors

Abstract

Deep learning has significantly improved the accuracy of precipitation nowcasting. However, most existing multimodal models typically use simple channel concatenation or interpolation methods for data fusion, which often overlook the feature differences between different modalities. This paper therefore proposes a novel precipitation nowcasting optimisation framework called FusionCast. This framework incorporates three types of data: historical precipitable water vapour (PWV) data derived from global navigation satellite system (GNSS) inversions, historical radar based quantitative precipitation estimation (QPE), and forecasted radar QPE serving as a future prior. The FusionCast model comprises two core modules: the future prior radar QPE processing Module, which forecasts future radar data; and the Radar PWV Fusion (RPF) module, which uses a gate mechanism to efficiently combine features from various sources. Experimental results show that FusionCast significantly improves nowcasting performance.
Paper Structure (28 sections, 8 equations, 7 figures, 2 tables)

This paper contains 28 sections, 8 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Overview of the Architecture
  5. Multimodal Encoders
  6. RPF Module
  7. PWV spatial attention
  8. Radar channel attention
  9. Cross-Modal Gated Fusion
  10. Recurrent Decoder with Autoregression
  11. Experiments
  12. Data Sources and Baseline
  13. Study Area
  14. PWV
  15. Radar QPE Products
  16. ...and 13 more sections

Figures (7)

  • Figure 1: (a) Incorporates sparse GNSS PWV stations, historical radar QPE data, and future radar QPE data as model inputs. (b) Different data types are encoded separately through a multi-branch architecture. (c) Information from diverse data sources is fused to enhance forecasting performance. (d) The final decoder generates forecast outputs, providing projections of future precipitation.
  • Figure 2: (a) A spatial attention mechanism is applied to PWV features to emphasise critical spatial information. (b) The integration of GNSS and radar data through feature fusion via a gating mechanism is demonstrated. (c) Channel attention mechanisms extract key features from radar data to optimise information representation.
  • Figure 3: (a) The geographic location of the study area (red bounding box) within the Contiguous United States. (b) Zoomed-in view showing the spatial distribution of the GNSS stations (red triangles) used in this work.
  • Figure 4: Comparison: Interpolated Image vs. Ground Truth in the PWV Dataset at 2023-02-01 00:00:00 with 496 Valid Stations. (left) Interpolated Image, (right) Ground Truth.
  • Figure 5: Visualisation comparison of precipitation intensity forecast results for the Mississippi River basin over the next two hours, at 12:40 UTC on 22 February 2023. From top to bottom: Ground Truth (GT), FusionCast (Ours), NowcastNet, Optical Flow, ConvLSTM and PredRNN. The highlighted area indicates where our model demonstrates significant improvement over all other models. FusionCast preserves both the spatial structure and intensity of precipitation more consistently, especially at longer lead times.
  • ...and 2 more figures