A Foundation Model for the Earth System

TL;DR

Aurora introduces a 1.3B-parameter 3D foundation model for the Earth system, trained on over a million hours of heterogeneous data and capable of fine-tuning across diverse domains such as air quality, ocean waves, tropical cyclones, and high-resolution weather. Its architecture combines a 3D Perceiver encoder, a deep multi-scale 3D Swin Transformer U-Net backbone, and a 3D Perceiver decoder to assemble a latent 3D representation that can be evolved autoregressively and reconstructed for various targets and resolutions. Across domains, Aurora matches or surpasses specialized operational systems while delivering orders of magnitude faster predictions and enabling cost-efficient, domain-adaptable forecasting. The work highlights the value of data diversity and scalable pretraining, with roll-out fine-tuning and LoRA enabling efficient long-horizon predictions and practical deployment considerations.

Abstract

Reliable forecasts of the Earth system are crucial for human progress and safety from natural disasters. Artificial intelligence offers substantial potential to improve prediction accuracy and computational efficiency in this field, however this remains underexplored in many domains. Here we introduce Aurora, a large-scale foundation model for the Earth system trained on over a million hours of diverse data. Aurora outperforms operational forecasts for air quality, ocean waves, tropical cyclone tracks, and high-resolution weather forecasting at orders of magnitude smaller computational expense than dedicated existing systems. With the ability to fine-tune Aurora to diverse application domains at only modest computational cost, Aurora represents significant progress in making actionable Earth system predictions accessible to anyone.

Figures (45)

• Figure 1: Aurora is a 1.3 billion parameter foundation model for the Earth system.a: Aurora is pretrained on multiple heterogeneous datasets with different resolutions, variables, and pressure levels. The model is then fine-tuned for multiple operational forecasting scenarios at different resolutions: atmospheric chemistry and air quality at $0.4^\circ$, wave modelling at $0.25^\circ$, hurricane tracking at $0.25^\circ$, and weather forecasting at $0.1^\circ$. b: Aurora is a flexible 3D Swin Transformer with 3D Perceiver-based encoders and decoders. The model is able to ingest inputs with different spatial resolutions, numbers of pressure levels, and variables.
• Figure 2: In an operational setting, Aurora matches or outperforms CAMS in most comparisons, at orders of magnitude smaller computational expense.a: Predictions for tropical cyclones NO${}_2$ by Aurora accurately predict CAMS analysis. Predicting atmospheric gases correctly is extremely challenging due to their spatially heterogeneous nature. In particular, NO${}_2$, like most air pollution variables, is skewed towards high values in areas with large anthropogenic emissions such as densely populated regions of East Asia. In addition, NO${}_2$ exhibits a strong diurnal cycle; e.g., sunlight reduces background levels of NO${}_2$ through a process called photolysis. Aurora accurately captures both the extremes and background levels. Aurora and CAMS forecasts are initialised with CAMS analysis on 1 Sep 2022 at 00 UTC. b: Across all lead times, Aurora matches or outperforms CAMS on 74% of all targets. c: At a lead time of three days, Aurora matches or outperforms CAMS on 89% of all variables. See \ref{['app:cams:full-results']} for the full results.
• Figure 3: In an operational setting, Aurora matches or outperforms HRES-WAM in the majority of comparisons.a: Aurora accurately predicts significant wave height and mean wave direction for Typhoon Nanmadol, the most intense tropical cyclone in 2022. The red box shows the location of the typhoon and the number is the peak significant wave height. Aurora's prediction and HRES-WAM Analysis are for 17 Sep 2022 at UTC 12, when Typhoon Nanmadol reached peak intensity. Aurora was initialised at 16 Sep 2022 at UTC 12. b: Across all lead times, Aurora matches or outperforms HRES-WAM on 86% of all wave variables. c: At a lead time of three days, Aurora matches or outperforms HRES-WAM on 91% of all surface-level variables. See \ref{['app:wave:full-results']} for the full results.
• Figure 4: In an operational setting, Aurora outperforms state-of-the-art tropical cyclone prediction systems for multiple agencies and regions worldwide.a: Aurora attains better track prediction MAE than multiple agencies in various regions. Official forecasts are given by OFCL, PGTW, CWA, BABJ, RJTD, RKSL, and BoM (bolded). For the North Atlantic and Eastern Pacific we additionally compare to various models utilised in creating OFCL (not bolded). A model does not always make forecasts, which means that different columns are computed over different data. Columns are therefore not indicative of model performance and only indicate the performance w.r.t. Aurora. Here "$\approx$" indicates that the 95% confidence interval for the cell contains zero (see \ref{['app:tcs:details-scorecard']} for details). On average, Aurora is 20% better than other agencies in the North Atlantic and East Pacific, 18% in the West Pacific, and 24% in the Australian region. b: On 21 July, a tropical depression intensified into a tropical storm and was named Typhoon Doksuri. Doksuri would become the costliest Pacific typhoon to date, inflicting more than 28 billion USD in damage. Aurora correctly predicts that Doksuri will make landfall in the Northern Philippines, whereas PGTW predicts that it will pass over Taiwan.
• Figure 5: In an operational setting, Aurora outperforms IFS HRES at 0.1$\degree$ in the vast majority of comparisons. Aurora is the only AI model to accurately estimate maximum 10m wind speed in storm Ciarán.a: Aurora outperforms IFS HRES on over 92% of targets. The scorecard is limited to pressure levels lower in the atmosphere due to restricted availability of test year data. b: Wind speed RMSE computed against measurements at weather stations. Aurora significantly outperforms IFS HRES. c: Operational predictions for Storm Ciarán compared to IFS HRES analysis at $0.1^\circ$. Black dots show the location of minimum MSL and therefore trace the path of the storm. The maximum 10m wind speed of the storm is shown in the bottom left corner of each prediction. To better facilitate the prediction of extreme events, Aurora was run without LoRA. See \ref{['section:sm-ciaran']}. d: Operational predictions for maximum 10m wind speed during Storm Ciarán by Aurora, FourCastNet, GraphCast, and Pangu-Weather. Aurora is able to predict the sudden increase in 10m wind speed, unlike the other AI models. The numbers for all AI models except Aurora have been extracted from Figure 3 by charltonperez2024ciaran.