Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Learning and Foundation Models for Weather Prediction: A Survey

Jimeng Shi, Azam Shirali, Bowen Jin, Sizhe Zhou, Wei Hu, Rahuul Rangaraj, Shaowen Wang, Jiawei Han, Zhaonan Wang, Upmanu Lall, Yanzhao Wu, Leonardo Bobadilla, Giri Narasimhan

TL;DR

This survey assesses how deep learning and foundation models are transforming weather prediction by contrasting data-driven approaches with physics-based numerical models. It organizes methods into three training paradigms—deterministic predictive learning, probabilistic generative learning, and pre-training with fine-tuning—and examines backbones such as Transformers, Graph Neural Networks, and diffusion/GAN architectures, across general-purpose and domain-specific contexts. The paper catalogues applications (precipitation, air quality, SST, flood, drought, tropical cyclones, wildfire), datasets, and open-source resources, and outlines a roadmap emphasizing trustworthy AI, retrieval-augmented models, weather-constrained generation, multi-modal learning, and data management. Its key contribution lies in offering a comprehensive taxonomy, up-to-date syntheses of models and datasets, and a forward-looking agenda to bridge research with practical, reliable, and scalable weather prediction. The findings underscore the potential of diffusion and foundation-model approaches to provide calibrated uncertainty and transferability, while also highlighting challenges in data quality, resolution mismatch, and computational demands for large-scale pre-trained weather models.

Abstract

Physics-based numerical models have been the bedrock of atmospheric sciences for decades, offering robust solutions but often at the cost of significant computational resources. Deep learning (DL) models have emerged as powerful tools in meteorology, capable of analyzing complex weather and climate data by learning intricate dependencies and providing rapid predictions once trained. While these models demonstrate promising performance in weather prediction, often surpassing traditional physics-based methods, they still face critical challenges. This paper presents a comprehensive survey of recent deep learning and foundation models for weather prediction. We propose a taxonomy to classify existing models based on their training paradigms: deterministic predictive learning, probabilistic generative learning, and pre-training and fine-tuning. For each paradigm, we delve into the underlying model architectures, address major challenges, offer key insights, and propose targeted directions for future research. Furthermore, we explore real-world applications of these methods and provide a curated summary of open-source code repositories and widely used datasets, aiming to bridge research advancements with practical implementations while fostering open and trustworthy scientific practices in adopting cutting-edge artificial intelligence for weather prediction. The related sources are available at https://github.com/JimengShi/ DL-Foundation-Models-Weather.

Deep Learning and Foundation Models for Weather Prediction: A Survey

TL;DR

This survey assesses how deep learning and foundation models are transforming weather prediction by contrasting data-driven approaches with physics-based numerical models. It organizes methods into three training paradigms—deterministic predictive learning, probabilistic generative learning, and pre-training with fine-tuning—and examines backbones such as Transformers, Graph Neural Networks, and diffusion/GAN architectures, across general-purpose and domain-specific contexts. The paper catalogues applications (precipitation, air quality, SST, flood, drought, tropical cyclones, wildfire), datasets, and open-source resources, and outlines a roadmap emphasizing trustworthy AI, retrieval-augmented models, weather-constrained generation, multi-modal learning, and data management. Its key contribution lies in offering a comprehensive taxonomy, up-to-date syntheses of models and datasets, and a forward-looking agenda to bridge research with practical, reliable, and scalable weather prediction. The findings underscore the potential of diffusion and foundation-model approaches to provide calibrated uncertainty and transferability, while also highlighting challenges in data quality, resolution mismatch, and computational demands for large-scale pre-trained weather models.

Abstract

Physics-based numerical models have been the bedrock of atmospheric sciences for decades, offering robust solutions but often at the cost of significant computational resources. Deep learning (DL) models have emerged as powerful tools in meteorology, capable of analyzing complex weather and climate data by learning intricate dependencies and providing rapid predictions once trained. While these models demonstrate promising performance in weather prediction, often surpassing traditional physics-based methods, they still face critical challenges. This paper presents a comprehensive survey of recent deep learning and foundation models for weather prediction. We propose a taxonomy to classify existing models based on their training paradigms: deterministic predictive learning, probabilistic generative learning, and pre-training and fine-tuning. For each paradigm, we delve into the underlying model architectures, address major challenges, offer key insights, and propose targeted directions for future research. Furthermore, we explore real-world applications of these methods and provide a curated summary of open-source code repositories and widely used datasets, aiming to bridge research advancements with practical implementations while fostering open and trustworthy scientific practices in adopting cutting-edge artificial intelligence for weather prediction. The related sources are available at https://github.com/JimengShi/ DL-Foundation-Models-Weather.
Paper Structure (65 sections, 10 equations, 3 figures, 3 tables)

This paper contains 65 sections, 10 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background and Preliminaries
  3. Weather Data Representation
  4. Station-Based Observation Data.
  5. Gridded Reanalysis Data.
  6. Weather Prediction Formulation
  7. Preliminaries
  8. Overview and Taxonomy
  9. Predictive Learning
  10. General-Purpose Large Models
  11. Transformer-based models.
  12. GNN-based models.
  13. Physics-AI-based models.
  14. Domain-Specific Models
  15. Transformer-based models.
  16. ...and 50 more sections

Figures (3)

  • Figure 1: Perspectives of weather forecasting.
  • Figure 2: The illustration of various frameworks of training deep learning models on weather prediction. For clarity, this visualization focuses exclusively on single-step predictions for a single variable.
  • Figure 3: A comprehensive taxonomy of deep learning and foundation models for weather prediction from the perspectives of training paradigms (dark yellow), model scopes (purple), and model architectures (pink).

Theorems & Definitions (3)

  • Definition 2.1: General-Purpose Large Models
  • Definition 2.2: Domain-Specific Models
  • Definition 2.3: Foundation Models