Extreme Precipitation Nowcasting using Transformer-based Generative Models

TL;DR

An innovative approach to extreme precipitation nowcasting by employing Transformer-based generative models, namely NowcastingGPT with Extreme Value Loss (EVL) regularization, which introduces a novel method for computing EVL without assuming fixed extreme representations.

Abstract

This paper presents an innovative approach to extreme precipitation nowcasting by employing Transformer-based generative models, namely NowcastingGPT with Extreme Value Loss (EVL) regularization. Leveraging a comprehensive dataset from the Royal Netherlands Meteorological Institute (KNMI), our study focuses on predicting short-term precipitation with high accuracy. We introduce a novel method for computing EVL without assuming fixed extreme representations, addressing the limitations of current models in capturing extreme weather events. We present both qualitative and quantitative analyses, demonstrating the superior performance of the proposed NowcastingGPT-EVL in generating accurate precipitation forecasts, especially when dealing with extreme precipitation events. The code is available at \url{https://github.com/Cmeo97/NowcastingGPT}.

Figures (16)

• Figure 1: ROC Curve for extreme event detection. Thresholds between $0.5$ and $10$ for precipitation values are used to define an extreme event. NowcastingGPT-EVL had the highest AUC, outperforming all other baselines.
• Figure 2: The image shows the NowcastingGPT-EVL model architecture. The VQ-VAE Encoder and Decoder are depicted in red and green respectively. The Extreme tokens classifier is depicted in orange, it takes the predicted tokens as input from the transformer and outputs the probabilities $u_t$ used in the EVL loss. The dashed line indicates that the output of the Classifier is only used to optimize the transformer and not as input.
• Figure 3: Nowcasting of extreme precipitation scenarios. The generation is conditioned on $3$ previous timestamps with the task to predict the next $6$ lead times. There is a gap of $30$ minutes between each timestamp. Images are upsampled to $256\times256$ pixels.
• Figure 4: PCC metric evaluation over the 6 lead times. Each point represents the average value for a specific lead time over the whole dataset. Higher values represent better performance. NowcastingGPT-EVL outperforms all other models.
• Figure 5: MSE metric evaluation over the 6 lead times. Each point represents the average value for a specific lead time over the whole dataset. Lower values represent better performance. NowcastingGPT-EVL and TECO outperforms all other models for bigger lead times.