HealDA: Highlighting the importance of initial errors in end-to-end AI weather forecasts

TL;DR

HealDA addresses the challenge of ML-based data assimilation by providing an observation-only, background-free initial-condition generator that maps a 24-hour window of heterogeneous observations to ERA5-like states on the HPX64 grid. Its two-part architecture—an observation encoder and an HPX ViT backbone—produces plug-and-play analyses that initialize off-the-shelf ML forecast models (e.g., FCN3, Aurora, FengWu) without retraining them. Across multiple metrics, HealDA analyses are close to ERA5 for many fields, and HealDA-initialized forecasts exhibit similar error growth to ERA5-initialized runs, with a modest lead-time loss largely attributed to larger initial errors and large-scale overfitting revealed by spectral diagnostics. The work demonstrates that ML DA can be effectively decoupled from forecast models, achieving fast, scalable, and robust initial-condition generation, while identifying key avenues—notably increased training data and improved priors—to close remaining gaps with physics-based analyses.

Abstract

AI weather models now rival leading numerical weather prediction (NWP) systems in medium-range skill. However, almost all still rely on NWP data assimilation (DA) to provide initial conditions, tying them to expensive infrastructure and limiting the practical speed and accuracy gains of ML. More recently, ML-based DA systems have been proposed, which are often trained and evaluated end-to-end with a forecast model, making it difficult to assess the quality of their analysis fields. We introduce HealDA, a global ML-based DA system that maps a short window of satellite and conventional observations directly to a 1° atmospheric state on the HEALPix grid, using a smaller sensor suite than operational NWP and no background forecast at runtime. We treat HealDA strictly as a DA module: its analyses are used to initialize off-the-shelf ML forecast models without any fine-tuning of either. For a variety of off-the-shelf ML forecast models, including FourCastNet3 (FCN3), Aurora, and FengWu, HealDA-initialized forecasts lose less than one day of effective lead time when scored against ERA5. HealDA-initialized FCN3 ensembles similarly trail those of the ECMWF IFS ENS system by < 24 h. We find that forecast error growth in these models i unchanged from HealDA initialization, and the skill gap primarily arises from the larger initial error of the HealDA analysis. Spectral analysis reveals that this stems from overfitting to the large scales and upper-tropospheric fields. We also demonstrate that small changes in the verification setup can shift apparent skill by 12--24h, underscoring the need for consistent scoring. Taken together, these results clarify the current performance of ML-based DA systems and show that a relatively simple, background-free network can already provide initial conditions that are usable by state-of-the-art ML forecast models with only modest loss in medium-range skill.

Figures (22)

• Figure 1: End-to-end HealDA system and forecasting pipeline. Observations from various remote-sensing instruments (ATMS, MHS, etc.) and in-situ sources (radiosondes, buoys, etc.) in the time window $[t_0 - 21\,\mathrm{h},\, t_0 + 3\,\mathrm{h}]$ are processed by HealDA, which consists of an Observation Encoder (Obs Encoder) followed by an HPX ViT backbone, to produce an analysis state on the HPX grid at the target time $t_0$. This analysis can then serve as the initial condition for an external forecast model (e.g., FCN3), to produce a 10-day forecast $(t_0 + 6\,\mathrm{h}, \dots, t_0 + 240\,\mathrm{h})$.
• Figure 2: RMSE of HealDA analysis vs IFS Time series of global RMSE for both HealDA and IFS against ERA5 in the 2022 test period, computed every 6 hours (00/06/12/18 UTC). The original data is shown with reduced opacity to reduce noise, and the solid line represents the 7-day moving average.
• Figure 3: Probabilistic FCN3 skill with HealDA and ERA5 initial conditions. CRPS of FCN3 forecasts initialized by HealDA and ERA5, both verified against ERA5 on the HPX64 grid and averaged over 128 initial conditions at 06/18 UTC in 2022. The inset panels zoom into the 6-48 h lead time range.
• Figure 4: Probabilistic skill of HealDA-initialized FCN3 vs IFS ENS. CRPS of IFS ENS forecasts and FCN3 forecasts initialized from HealDA, verified against ERA5 on the HPX64 grid and averaged over 128 initial conditions at 00/12 UTC in 2022.
• Figure 5: Analysis error spectral decomposition. Spherical power spectra of HealDA and IFS HRES analysis errors on the HPX64 grid, scored relative to ERA5. The HealDA error spectra are shown, averaged over the test year (2022), in solid lines, and a year from the training period (2021), in dashed lines. For IFS, the error spectra averaged across 2021-2022 are shown. Spectra are shown as a function of spherical harmonic degree $\ell$ (large scales on the left, small scales on the right) and plotted as $10\log_{10} C_\ell$, where $C_\ell = \frac{1}{2\ell+1}\sum_{m=-\ell}^{\ell}\left|a_{\ell m}\right|^2$.