Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hybrid ensemble forecasting combining physics-based and machine-learning predictions through spectral nudging

Inna Polichtchouk, Simon Lang, Sarah-Jane Lock, Michael Maier-Gerber, Peter Dueben

TL;DR

The first application of spectral nudging in a probabilistic ensemble forecasting framework, combining the physics-based IFS-ENS with forecasts from the probabilistic machine-learned AIFS-ENS ensemble, shows substantial improvements in large-scale forecast skill.

Abstract

We present the first application of spectral nudging in a probabilistic ensemble forecasting framework, combining the physics-based ECMWF Integrated Forecasting System ensemble (IFS-ENS) with forecasts from the probabilistic machine-learned AIFS-ENS ensemble. Large scales of virtual temperature and vorticity are relaxed toward the machine-learned forecasts, while mesoscale structures remain governed by the physics-based model. This hybrid ensemble shows substantial improvements in large-scale forecast skill, with gains in predictive skill extended by up to two days in the tropics and by approximately half a day in the extra-tropics relative to IFS-ENS. Despite nudging being applied only to upper-air fields, improvements are also found in several near-surface parameters. Tropical cyclone track forecasts improve significantly, consistent with improved representation of the large-scale steering flow, without degrading storm intensity or ensemble spread. These results demonstrate that spectral nudging can be successfully extended to ensemble prediction and provide an attractive pathway for combining machine-learned and physics-based weather prediction systems.

Hybrid ensemble forecasting combining physics-based and machine-learning predictions through spectral nudging

TL;DR

The first application of spectral nudging in a probabilistic ensemble forecasting framework, combining the physics-based IFS-ENS with forecasts from the probabilistic machine-learned AIFS-ENS ensemble, shows substantial improvements in large-scale forecast skill.

Abstract

We present the first application of spectral nudging in a probabilistic ensemble forecasting framework, combining the physics-based ECMWF Integrated Forecasting System ensemble (IFS-ENS) with forecasts from the probabilistic machine-learned AIFS-ENS ensemble. Large scales of virtual temperature and vorticity are relaxed toward the machine-learned forecasts, while mesoscale structures remain governed by the physics-based model. This hybrid ensemble shows substantial improvements in large-scale forecast skill, with gains in predictive skill extended by up to two days in the tropics and by approximately half a day in the extra-tropics relative to IFS-ENS. Despite nudging being applied only to upper-air fields, improvements are also found in several near-surface parameters. Tropical cyclone track forecasts improve significantly, consistent with improved representation of the large-scale steering flow, without degrading storm intensity or ensemble spread. These results demonstrate that spectral nudging can be successfully extended to ensemble prediction and provide an attractive pathway for combining machine-learned and physics-based weather prediction systems.
Paper Structure (12 sections, 3 equations, 9 figures)

This paper contains 12 sections, 3 equations, 9 figures.

Table of Contents

  1. Introduction
  2. Models and Methods
  3. ECMWF IFS
  4. Probabilistic Model-Level AIFS-ENS
  5. Spectral nudging
  6. Experimental setup
  7. Results
  8. Example of hybrid forecast structure
  9. Large-scale skill scores
  10. Tropical cyclones
  11. Northern Hemisphere Extremes
  12. Summary, Discussion and Conclusions

Figures (9)

  • Figure 1: Global kinetic energy spectra as a function of total wavenumber at 500 hPa for forecast day 10 of the model-level AIFS-ENS-ML (dash-dotted black), IFS-ENS (solid blue), hy-IFS-ENS (dashed red) and operational AIFS-ENS (dotted green). Spectra are averaged for all eight perturbed members and for all forecasts initialized at 00 UTC between 1 July 2024 and 30 September 2024. Black vertical line shows total wavenumber 21, which is the cut-off wavenumber for nudging in hy-IFS-ENS.
  • Figure 2: 850 hPa meridional wind ($v$ component; m s$^{-1}$) at forecast day 4 for ensemble member one of (a) IFS-ENS, (b) hy-IFS-ENS, and (c) operational AIFS-ENS initialized on 20 January 2025 at 00 UTC.
  • Figure 3: Summary scorecard comparing the difference in forecast skill between the spectrally nudged IFS ensemble (hy-IFS-ENS) and the control IFS ensemble (IFS-ENS), using the fair continuous ranked probability score (FCRPS) and the anomaly correlation of the ensemble mean (CCAF), verified against ECMWF operational analysis (upper half) and against radiosonde and SYNOP station observations (lower half). Each box corresponds to a 24-h lead-time interval from forecast day 1 to day 15. Shades of blue indicate improvement in hy-IFS-ENS relative to IFS, and shades of red indicate degradation. Colour saturation reflects the magnitude of the normalized difference. Statistically significant differences at the 99.7% confidence level are outlined with coloured frames. Regional abbreviations denote Northern Hemisphere (n.hem; 20$\degree$N-90$\degree$N), Southern Hemisphere (s.hem; 20$\degree$S–90$\degree$S), Tropics (20$\degree$N-20$\degree$S) and Europe (35$\degree$N-75$\degree$N, 12.5$\degree$W-42.5$\degree$E). Scores are computed using 8 perturbed members from 447 forecasts initialized at 00 and 12 UTC between 1 July 2024 and 28 February 2025.
  • Figure 4: Fair CRPS for (a,b,c) 850 hPa temperature and (d,e,f) 500 hPa wind speed for hy-IFS-ENS (dashed red), IFS-ENS (solid blue) and operational AIFS-ENS (dotted green) over the (a,d) Northern Hemisphere, (b,e) tropics and (c,f) Southern Hemisphere. For differences between IFS-ENS and hy-IFS-ENS, the data here correspond to the lines labelled "t850" and "ff500" in Figure \ref{['fig:fig2']}. Smaller FCRPS values indicate higher skill. Scores are computed against ECMWF operational analysis from over 447 forecasts initialized at 00 UTC and 12 UTC between 1 July 2024 and 28 February 2025.
  • Figure 5: Fair CRPS of (a) 10-m wind speed, (b) 2-m temperature and (c) total precipitation for hy-IFS-ENS (dashed red), IFS-ENS (solid blue) and operational AIFS-ENS (dotted green) over the Northern Hemisphere. For differences between IFS-ENS and AIFS-ENS, the data here correspond to the lines labelled "10ff", "2t" and "tp" in Figure \ref{['fig:fig2']}. Scores are computed against SYNOP surface observations from over 447 forecasts initialized at 00 UTC and 12 UTC between 1 July 2024 and 28 February 2025.
  • ...and 4 more figures