FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead

TL;DR

FengWu reframes global medium-range weather forecasting as a multimodal, multi-task learning problem, treating each atmospheric variable as a distinct modality and fusing them with a cross-modal Transformer. It integrates modal-specific encoders/decoders, an uncertainty-based loss to automatically balance predictions across variables and levels, and a replay buffer to enable efficient long-lead autoregressive training. Trained on 39 years of ERA5 data at 0.25° with 37 vertical levels (189 predictands), it achieves state-of-the-art skill, outperforming GraphCast on 80% of 880 targets and pushing the skillful lead time to 10.75 days (ACC > 0.6 for z500). Inference is fast on modern accelerators, and the replay buffer substantially improves long-lead forecasts, highlighting the practical potential of AI-driven NWP while acknowledging initialization-related fairness considerations when comparing to physics-based baselines.

Abstract

We present FengWu, an advanced data-driven global medium-range weather forecast system based on Artificial Intelligence (AI). Different from existing data-driven weather forecast methods, FengWu solves the medium-range forecast problem from a multi-modal and multi-task perspective. Specifically, a deep learning architecture equipped with model-specific encoder-decoders and cross-modal fusion Transformer is elaborately designed, which is learned under the supervision of an uncertainty loss to balance the optimization of different predictors in a region-adaptive manner. Besides this, a replay buffer mechanism is introduced to improve medium-range forecast performance. With 39-year data training based on the ERA5 reanalysis, FengWu is able to accurately reproduce the atmospheric dynamics and predict the future land and atmosphere states at 37 vertical levels on a 0.25° latitude-longitude resolution. Hindcasts of 6-hourly weather in 2018 based on ERA5 demonstrate that FengWu performs better than GraphCast in predicting 80\% of the 880 reported predictands, e.g., reducing the root mean square error (RMSE) of 10-day lead global z500 prediction from 733 to 651 $m^{2}/s^2$. In addition, the inference cost of each iteration is merely 600ms on NVIDIA Tesla A100 hardware. The results suggest that FengWu can significantly improve the forecast skill and extend the skillful global medium-range weather forecast out to 10.75 days lead (with ACC of z500 > 0.6) for the first time.

Figures (7)

• Figure 1: The skillful forecast lead time of FengWu on z500 and t2m. With the ACC > 0.6 as the criterion of a skillful weather prediciton system, our findings are that FengWu can push the skillful forecast lead times to 10.75 days and 11.5 days for z500 and t2m, respectively.
• Figure 2: Overview of FengWu's architecture. FengWu first treats the multiple weather factors as different modalities and extracts their feature embeddings independently. And then a transformer-based network is utilized to fuse and pass messages among different modalities. Finally, the high-level feature representation is used to get the predictors via the modal-customized decoder.
• Figure 3: Latitude-weighted RMSE skill of FengWu (red lines) and GraphCast (black lines) predicting the weather in 2018 (Lower RMSE is better). The x-axis in each sub-figure represents lead time, at a 6-hour interval over a 10-day lead time. The y-axis represents the latitude-weighted RMSE defined in Eq. \ref{['eq:MSE']}.
• Figure 4: ACC skill of FengWu and GraphCast predicting the weather in 2018 (Higher ACC is better). The x-axis in each sub-figure represents lead time, at a 6-hour interval over a 10-day lead time. The y-axis represents the ACC defined in Eq. \ref{['eq:ACC']}.
• Figure 5: Forecast images and absolute error for z500. Figures of z500 on days 3, 5, and 10 are presented with initialization time at 2018-02-11 00:00 UTC. The subtitles at the top of the columns indicate the dates of prediction. The first row and second row show FengWu and ERA5 ground truth, respectively. Row 3 shows the absolute error between FengWu and ERA5.