Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning

TL;DR

This work introduces SMARL to develop closures for canonical prototypes of atmospheric/oceanic turbulence, using only the enstrophy spectrum, estimated from a few high-fidelity samples, as reward, to enable stable simulations that reproduce high-fidelity simulation statistics and capture in particular extremes.

Abstract

Accurate subgrid-scale closures are essential for weather/climate models, where predicting extreme events is critical. Traditional closures have structural errors, e.g., producing excessive diffusion that dampens extremes. Artificial intelligence has gained attention for closure modeling, but the prediction of extreme events remains challenging. Supervised offline learning needs abundant high-fidelity training data and can lead to instabilities. Online learning algorithms are emerging as an alternative, but reliance on differentiable numerical solvers or scalable optimizers hinders broad use. Here, we introduce SMARL to develop closures for canonical prototypes of atmospheric/oceanic turbulence, using only the enstrophy spectrum, estimated from a few high-fidelity samples, as reward. This reward ensures that the model captures the cascades of scales in these simulations. These online-learned closures enable stable simulations, with up to five orders of magnitude fewer degrees of freedom, that reproduce high-fidelity simulation statistics and capture in particular extremes. We interpret the closures by analyzing the SMARL policy and demonstrate generalization to other flows. The results highlight SMARL as a potent tool for developing closures capable of capturing extremes in atmospheric/oceanic flows, opening new capabilities for effective climate modeling.

Figures (8)

• Figure 1: Training of SMARL for SGS closures: (1) reference data consists of 5 samples of a short DNS (\ref{['eq:NS1', 'eq:NS2']}); (2) online training of agents. The state-action map has input state $s'(t)$ (spectrum of enstrophy $\hat{Z}$ up to LES cutoff wavenumber $k_c$) and domain-averaged output action $c_l(t)$ (coefficient in the closure, \ref{['eq:tau_SGS']}) that maximizes the reward $r(t)$. During testing, the policy is coupled to the low-resolution numerical solver to produce a long LES that is $2000\times$ longer than the DNS training set and $1000\times$ the training horizon. Performance for extreme events is evaluated in terms of the vorticity PDF $\mathcal{P}(\omega)$, against DNS and dynamic physics-based SGS models.
• Figure 2: A-posteriori testing results in terms of the PDF of vorticity ($\mathcal{P}(\omega)$). The $x-$axis is normalized by the standard deviation of vorticity ($\sigma_\omega$) computed from DNS of each case. The shading area represents the uncertainty (25-75 quartiles) of the DNS PDF.
• Figure 3: Total enstrophy diffusion $\langle P_z\rangle$ obtained from the FDNS, RL-Leith, DSmag and DLeith for the four cases.
• Figure 4: First column: PDF of model coefficients $c$; second column: $S_k$; and third column: $S_{T_k}$. The error bars on the Sobol indices indicate 95% confidence level. The Nyquist (or cut-off) wavenumber $k_c$ is the maximum wavenumber (i.e., $k_c=15$ in Case 1 and 2, $k_c=61$ in Case 3, and $k_c=127$ in Case 4). The dash lines in second and third columns mark the forcing wavenumber $k_f$.
• Figure 5: Generalization testing of SMARL to higher ($15\times Re$) turbulence (SMARL trained on Case 1 and tested on Case 5). (a) shows the turbulent kinetic energy spectra, $\hat{E}(k)$, and (b) shows the PDF, $\mathcal{P}(\omega)$. In (b), the $x-$axis is normalized by the standard deviation of vorticity ($\sigma_\omega$) computed from DNS. The shading area represents the uncertainty (25-75 quartiles) of the DNS PDF.