AI Foundation Models for Weather and Climate: Applications, Design, and Implementation

TL;DR

The paper surveys the emergence of weather and climate foundation models built on transformer, graph, and operator-learning frameworks, arguing that a generalizable FM for Earth systems is nascent but feasible. It outlines applications from nowcasting to downscaling, data assimilation, and climate analysis, and discusses design principles, data needs, and evaluation metrics to guide development. Key contributions include a taxonomy of model components (Transformers, GNNs, Neural Operators), pretraining strategies (RMSE, contrastive, domain-specific tasks, diffusion), and a pragmatic roadmap emphasizing multi-scale robustness and long-term stability. The authors advocate starting with scalable, multi-scale weather FM focused on nowcasting to medium-range forecasts, while highlighting challenges in data volume, grid representations, and rollout fidelity that must be addressed for operational deployment.

Abstract

Machine learning and deep learning methods have been widely explored in understanding the chaotic behavior of the atmosphere and furthering weather forecasting. There has been increasing interest from technology companies, government institutions, and meteorological agencies in building digital twins of the Earth. Recent approaches using transformers, physics-informed machine learning, and graph neural networks have demonstrated state-of-the-art performance on relatively narrow spatiotemporal scales and specific tasks. With the recent success of generative artificial intelligence (AI) using pre-trained transformers for language modeling and vision with prompt engineering and fine-tuning, we are now moving towards generalizable AI. In particular, we are witnessing the rise of AI foundation models that can perform competitively on multiple domain-specific downstream tasks. Despite this progress, we are still in the nascent stages of a generalizable AI model for global Earth system models, regional climate models, and mesoscale weather models. Here, we review current state-of-the-art AI approaches, primarily from transformer and operator learning literature in the context of meteorology. We provide our perspective on criteria for success towards a family of foundation models for nowcasting and forecasting weather and climate predictions. We also discuss how such models can perform competitively on downstream tasks such as downscaling (super-resolution), identifying conditions conducive to the occurrence of wildfires, and predicting consequential meteorological phenomena across various spatiotemporal scales such as hurricanes and atmospheric rivers. In particular, we examine current AI methodologies and contend they have matured enough to design and implement a weather foundation model.

Figures (1)

• Figure 1: Meteorology and climate science perspective on foundation models, with spatiotemporal windows of focus. WeatherFM is the most mature, but as indicated, current state-of-the-art AI models using operator learning, transformers, and graphs are mostly around the red ERA5 spatiotemporal line within the broader window. Scopes of FM: Earth System (blue dots/box); nowcasting (orange dash/box); weather (black dash, blue box); climate (yellow dash/box).