Incorporating Multivariate Consistency in ML-Based Weather Forecasting with Latent-space Constraints

TL;DR

The authors address blurring and physical-inconsistency in ML-based weather forecasts by reframing rollout training as a weak-constraint 4DVar problem and replacing model-space loss with latent-space constraints derived from an autoencoder. By approximating the reanalysis error covariance in latent space as near-diagonal, they implement a tractable loss that preserves multivariate dependencies and scale interactions, improving long-range skill and fine-scale structure while maintaining physical realism. Experiments on a coarsened ERA5 dataset show that latent-space constrained models (DFM-LC) outperform model-space constrained counterparts (DFM-MC) in multiscale fidelity and dynamical balance, though with computational cost and some long-range RMSE trade-offs. The framework further extends to heterogeneous data sources, offering a unified objective for integrating reanalysis and observations in deterministic forecast training, with potential extensions to probabilistic forecasts and Earth-system coupling.

Abstract

Data-driven machine learning (ML) models have recently shown promise in surpassing traditional physics-based approaches for weather forecasting, leading to a so-called second revolution in weather forecasting. However, most ML-based forecast models treat reanalysis as the truth and are trained under variable-specific loss weighting, ignoring their physical coupling and spatial structure. Over long time horizons, the forecasts become blurry and physically unrealistic under rollout training. To address this, we reinterpret model training as a weak-constraint four-dimensional variational data assimilation (WC-4DVar) problem, treating reanalysis data as imperfect observations. This allows the loss function to incorporate reanalysis error covariance and capture multivariate dependencies. In practice, we compute the loss in a latent space learned by an autoencoder (AE), where the reanalysis error covariance becomes approximately diagonal, thus avoiding the need to explicitly model it in the high-dimensional model space. We show that rollout training with latent-space constraints improves long-term forecast skill and better preserves fine-scale structures and physical realism compared to training with model-space loss. Finally, we extend this framework to accommodate heterogeneous data sources, enabling the forecast model to be trained jointly on reanalysis and multi-source observations within a unified theoretical formulation.

Figures (7)

• Figure 1: Training deterministic forecast models with (a) model‑space constraints and (b) latent‑space constraints. The superscript $a$ on model states $\boldsymbol{x}$ and latent states $\boldsymbol{z}$ indicates reanalysis data, while the subscript $i$ denotes the $i$‑th step during rollout training. $\mathcal{E}$ and $\mathcal{D}$ denote the encoder and decoder of the pretrained autoencoder for reanalysis. The bottom panel shows T500 forecasts initialized from ERA5 reanalysis at 00 UTC 1 Jan 2020, with black boxes highlighting differences in fine‑scale structures at the 15‑day lead.
• Figure 2: Comparison of the globally averaged forecast error of DFM-LC, DFM-MC, and the forecast model trained without rollout. Forecasts are initialized twice daily from ERA5 reanalysis throughout 2020 and evaluated against ERA5 using latitude-weighted root mean square error (RMSE).
• Figure 3: Zonal power spectra of forecast fields from DFM-LC and DFM-MC at different lead times. Zonal-mean power spectra of (a) Z500 and (b) T850 over the midlatitudes (30°–60° N/S), computed from forecasts at day 1 (solid lines) and day 15 (dashed lines), and compared with ERA5.
• Figure 4: Spin-up forecasts of specific humidity at 500 hPa (Q500) from (a) DFM-LC and (b) DFM-MC, initialized from a smoothed initial state. The initial condition is generated by applying bicubic interpolation to a 25× coarsened ERA5 analysis at 00 UTC on 1 January 2020.
• Figure 5: Geostrophic balance diagnostics in forecasts from DFM-LC and DFM-MC.(a) Zonal and meridional components of actual wind $(u, v)$ and geostrophic wind $(u_g, v_g)$ at 500 hPa over the Northern Hemisphere midlatitudes (30°–60°N), derived from ERA5 reanalysis at 00 UTC on 1 January 2020. (b) Time evolution of the geostrophic imbalance ratio $R_{\mathrm{imb}}$ over 30-day forecasts from DFM-LC and DFM-MC, averaged over forecasts initialized daily at 00 UTC throughout 2020. (c) Zonal distribution of the difference between actual and geostrophic zonal wind ($u - u_g$) at 500 hPa for forecasts initialized on 1 January 2020, shown at selected lead times (days 1, 10, 20, and 30) for DFM-LC and DFM-MC, with ERA5 shown for reference.