Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Assessing the ability of a stretched-grid deep-learning weather prediction model to capture physical balances

Francesco Pasquini, Michiel Baatsen, Bastien François, Natalie Theeuwes, Maurice Schmeits

Abstract

Weather forecasting has traditionally relied on Numerical Weather Prediction (NWP) models, which simulate weather by solving the governing fluid equations. Recently, the emergence of Deep Learning Weather Prediction (DLWP) models has opened a new era in weather forecasting, offering a data-driven alternative to classical NWP approaches. Regional DLWP models such as the stretched-grid model Bris developed by Met Norway, have demonstrated performance on par with, or even slightly better than regional NWP models across a range of standard forecast metrics. By overcoming the coarse horizontal resolution that constrained earlier global data-driven models, the operational use of regional DLWP systems now appears increasingly promising. Nevertheless, the performance of such models during extreme events is generally inferior to that of regional NWP models, and comprehensive evaluations of their ability to generate physically realistic forecasts are still lacking. Here, we present a study comparing the physical consistency of the deterministic version of Bris with the control run of the operational MetCoOp Ensemble Prediction System (MEPS) in forecasting the severe extratropical cyclone Poly, which hit the Netherlands on 5 July 2023. We examine whether Bris accurately represents deviations from key atmospheric balances and whether it reproduces expected dynamics of the storm. We show that, despite its relatively good performance in terms of RMSE, Bris struggles to capture important mesoscale features of the event and that it significantly disrupts several atmospheric balances. This unrealistic disruption is mainly linked to the fine-scale noise evidenced in its output fields, which leads to incorrect and unrealistic spatial gradients. These results raise critical questions for improving AI-based models to better represent extreme events and how to ensure physical consistency in their predictions.

Assessing the ability of a stretched-grid deep-learning weather prediction model to capture physical balances

Abstract

Weather forecasting has traditionally relied on Numerical Weather Prediction (NWP) models, which simulate weather by solving the governing fluid equations. Recently, the emergence of Deep Learning Weather Prediction (DLWP) models has opened a new era in weather forecasting, offering a data-driven alternative to classical NWP approaches. Regional DLWP models such as the stretched-grid model Bris developed by Met Norway, have demonstrated performance on par with, or even slightly better than regional NWP models across a range of standard forecast metrics. By overcoming the coarse horizontal resolution that constrained earlier global data-driven models, the operational use of regional DLWP systems now appears increasingly promising. Nevertheless, the performance of such models during extreme events is generally inferior to that of regional NWP models, and comprehensive evaluations of their ability to generate physically realistic forecasts are still lacking. Here, we present a study comparing the physical consistency of the deterministic version of Bris with the control run of the operational MetCoOp Ensemble Prediction System (MEPS) in forecasting the severe extratropical cyclone Poly, which hit the Netherlands on 5 July 2023. We examine whether Bris accurately represents deviations from key atmospheric balances and whether it reproduces expected dynamics of the storm. We show that, despite its relatively good performance in terms of RMSE, Bris struggles to capture important mesoscale features of the event and that it significantly disrupts several atmospheric balances. This unrealistic disruption is mainly linked to the fine-scale noise evidenced in its output fields, which leads to incorrect and unrealistic spatial gradients. These results raise critical questions for improving AI-based models to better represent extreme events and how to ensure physical consistency in their predictions.

Paper Structure

This paper contains 16 sections, 10 equations, 13 figures.

Table of Contents

  1. Introduction
  2. Models and Case Study
  3. Data and Methodology
  4. Hydrostatic Balance
  5. Geostrophic Balance
  6. Gradient Wind Balance
  7. Continuity Equation
  8. Results
  9. A first general assessment
  10. Hydrostatic balance
  11. Geostrophic balance
  12. Gradient wind balance
  13. Continuity equation
  14. Fine-scale noise and spectral analysis
  15. Discussion
  16. ...and 1 more sections

Figures (13)

  • Figure 1: Maps of 10m wind speed forecasts (color) and mean sea-level pressure (blue contours, in hPa) at +6h lead time for the MEPS (NWP; left) and Bris (DDM; centre) model, together with the corresponding analysis fields (right). Fields are provided for four different times during the occurrence of windstorm Poly: 00, 06, 12, 18UTC on 5 July 2023 (from top to bottom).
  • Figure 2: RMSE of (a) 10m wind speed, (b) 500hPa wind speed, (c) mean sea-level pressure, (d) 500hPa geopotential in the selected 443x443 domain (Fig. S1). Values are given as a function of lead time for four times (00, 06, 12, 18 UTC) on 5 July 2023, during the occurrence of windstorm Poly.
  • Figure 3: MEPS (NWP; left) and Bris (DDM; centre) +6h forecasts, along with the corresponding analysis fields (right) at 06 UTC on 5 July 2023 (windstorm Poly at its mature stage), of: (a–c) 500 hPa specific humidity (shading) and geopotential (blue contours, in $10^2\,\mathrm{m^2\,s^{-2}}$); (d–f) 500hPa temperature anomalies; (g–i) 500hPa vertical velocity; and (j–l) hydrostatic balance anomalies between 500hPa and 400hPa. Anomalies are computed relative to the domain-mean values of each field over the 443x443 domain shown in the upper left corner of panels (d)-(f) and (j)-(l)
  • Figure 4: Spatial median normalized ageostrophic wind speeds in MEPS (NWP) forecast, Bris (DDM) predictions, and analysis fields, shown as functions of lead time and computed at locations where Ro<0.1 within the selected 443×443 domain. Results refer to 500hPa winds at four times (00, 06, 12, 18 UTC) on 5 July 2023, when windstorm Poly moved over the domain. Ageostrophic components are normalized by the mean wind speed of the distribution and are presented for three cases: f1 (no smoothing), f7 and f13 (winds computed after smoothing; see section \ref{['sec3']}).
  • Figure 5: Scatter plots of 500hPa predicted wind speed versus geostrophic wind speed for (a–c) unsmoothed fields (f1), (d–f) winds computed after smoothing (f13), and (g–i) noise-to-noise comparisons. Values are given for MEPS (NWP; left), Bris (DDM; centre) +6h forecasts, and the corresponding analysis (right) at 06 UTC on 5 July 2023, during the occurrence of windstorm Poly, considering only grid points with Ro<0.1 within the selected 443 $\times$ 443 domain. The dashed black line (y = x) indicates perfect agreement.
  • ...and 8 more figures