Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Subseasonal Weather Forecast using Teleconnection-informed Transformers

Shan Zhao, Zhitong Xiong, Xiao Xiang Zhu

TL;DR

This work tackles subseasonal weather forecasting by introducing a teleconnection-informed transformer that ingests Ocean Climate Indices (OCIs) through a specialized temporal filter and builds on a pretrained 3D Earth-specific Transformer (Pangu) with LoRA-based parameter-efficient fine-tuning. By updating only a small fraction of parameters, the method extends predictive horizons to about $h=336$ hours across multiple surface and upper-air variables while enhancing spatial saliency and physical consistency. The approach demonstrates improved predictive skill and output granularity over baselines, highlighting the importance of teleconnections in shaping subseasonal dynamics and offering a resource-efficient path to leverage foundation models for diverse downstream tasks.

Abstract

Subseasonal forecasting, which is pivotal for agriculture, water resource management, and early warning of disasters, faces challenges due to the chaotic nature of the atmosphere. Recent advances in machine learning (ML) have revolutionized weather forecasting by achieving competitive predictive skills to numerical models. However, training such foundation models requires thousands of GPU days, which causes substantial carbon emissions and limits their broader applicability. Moreover, ML models tend to fool the pixel-wise error scores by producing smoothed results which lack physical consistency and meteorological meaning. To deal with the aforementioned problems, we propose a teleconnection-informed transformer. Our architecture leverages the pretrained Pangu model to achieve good initial weights and integrates a teleconnection-informed temporal module to improve predictability in an extended temporal range. Remarkably, by adjusting 1.1% of the Pangu model's parameters, our method enhances predictability on four surface and five upper-level atmospheric variables at a two-week lead time. Furthermore, the teleconnection-filtered features improve the spatial granularity of outputs significantly, indicating their potential physical consistency. Our research underscores the importance of atmospheric and oceanic teleconnections in driving future weather conditions. Besides, it presents a resource-efficient pathway for researchers to leverage existing foundation models on versatile downstream tasks.

Efficient Subseasonal Weather Forecast using Teleconnection-informed Transformers

TL;DR

This work tackles subseasonal weather forecasting by introducing a teleconnection-informed transformer that ingests Ocean Climate Indices (OCIs) through a specialized temporal filter and builds on a pretrained 3D Earth-specific Transformer (Pangu) with LoRA-based parameter-efficient fine-tuning. By updating only a small fraction of parameters, the method extends predictive horizons to about hours across multiple surface and upper-air variables while enhancing spatial saliency and physical consistency. The approach demonstrates improved predictive skill and output granularity over baselines, highlighting the importance of teleconnections in shaping subseasonal dynamics and offering a resource-efficient path to leverage foundation models for diverse downstream tasks.

Abstract

Subseasonal forecasting, which is pivotal for agriculture, water resource management, and early warning of disasters, faces challenges due to the chaotic nature of the atmosphere. Recent advances in machine learning (ML) have revolutionized weather forecasting by achieving competitive predictive skills to numerical models. However, training such foundation models requires thousands of GPU days, which causes substantial carbon emissions and limits their broader applicability. Moreover, ML models tend to fool the pixel-wise error scores by producing smoothed results which lack physical consistency and meteorological meaning. To deal with the aforementioned problems, we propose a teleconnection-informed transformer. Our architecture leverages the pretrained Pangu model to achieve good initial weights and integrates a teleconnection-informed temporal module to improve predictability in an extended temporal range. Remarkably, by adjusting 1.1% of the Pangu model's parameters, our method enhances predictability on four surface and five upper-level atmospheric variables at a two-week lead time. Furthermore, the teleconnection-filtered features improve the spatial granularity of outputs significantly, indicating their potential physical consistency. Our research underscores the importance of atmospheric and oceanic teleconnections in driving future weather conditions. Besides, it presents a resource-efficient pathway for researchers to leverage existing foundation models on versatile downstream tasks.
Paper Structure (9 sections, 4 equations, 2 figures, 2 tables)

This paper contains 9 sections, 4 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Teleconnection-informed temporal filter
  4. Parameter efficient adaptation
  5. Experimental validation
  6. Data
  7. Evaluation
  8. Results
  9. Conclusion

Figures (2)

  • Figure 1: Workflow of teleconnection-informed transformers. The input signals are upper-air variables, surface variables, and OCIs. The OCIs are processed by the temporal module with various kernel sizes to filter the features of climate variables. The 3D Earth-Specific Transformers (Pangu) take the pre-trained weights at 24h lead time and use the LoRA strategy to achieve predictability over longer horizons.
  • Figure 2: Example visualizations of subseasonal forecast of u10m by proposed method and baselines. From left to right are the input frame at 2018-10-01 00:00 UTC, target frame at 2018-10-15 00:00 UTC, prediction, and bias between the prediction and the target frame.