The High-Frequency and Rare Events Barriers to Neural Closures of Atmospheric Dynamics

TL;DR

The atmospheric 1980 Lorenz model, a simplified version of the Primitive Equations that drive climate models, serves as a compelling example because it captures the essence of the difficulties neural networks may experience in capturing rare events due to limitations in how data is sampled.

Abstract

Recent years have seen a surge in interest for leveraging neural networks to parameterize small-scale or fast processes in climate and turbulence models. In this short paper, we point out two fundamental issues in this endeavor. The first concerns the difficulties neural networks may experience in capturing rare events due to limitations in how data is sampled. The second arises from the inherent multiscale nature of these systems. They combine high-frequency components (like inertia-gravity waves) with slower, evolving processes (geostrophic motion). This multiscale nature creates a significant hurdle for neural network closures. To illustrate these challenges, we focus on the atmospheric 1980 Lorenz model, a simplified version of the Primitive Equations that drive climate models. This model serves as a compelling example because it captures the essence of these difficulties.

Figures (10)

• Figure 1: Panel A: Illustration, for the $z_3$-variable, of the BE manifold's ability in capturing the L80 model's slow motion. See Gent_McWilliams82 and Appendix \ref{['Sect_Appendix_BE']} for a derivation. Panels (B) and (C): Neural parameterizations $\boldsymbol{\mathcal{X}}_{\boldsymbol{\theta}}^3$ for the $x_3$-variable, as learnt through random selection (NN$_1$)/predefined selection (NN$_2$). Visualized here as mappings from $(y_1,y_2)$ onto the unit sphere in $\mathbb{R}^3$. Panels (D): Same visualization adopted for the BE manifold. Panels (E): High-frequency residual $E_{NN_1}(t)$ for $x_3$ (black) given by \ref{['Eq_residual_err']} and its difference with $E_{NN_2}(t)$ (red).
• Figure 2: False quasiperiodicity produced by a slow neural closure. Here, the slow neural closure Eq. \ref{['Eq_low-passNN']} is driven by $\boldsymbol{\mathcal{Z}}_{\boldsymbol{\theta}}$ and $\boldsymbol{\mathcal{X}}_{\boldsymbol{\theta}}$ that are trained using a low-pass filtered version of $\boldsymbol{y}(t)$ (blue curve in Panel (A)) unlike the closure defined in Eq. \ref{['Eq_low-passNN']} where the slow neural closure is trained using the unfiltered $\boldsymbol{y}$-variable.
• Figure 3: Sensitivity of the slow neural closures. Here, NN$_1$ and NN$_2$ differ only in their training modalities. NN$_1$ is learnt from random selection of the training, validation, and testing sets, and NN$_2$ from a predefined selection with the same aspect ratios; see Text. The corresponding loss functions differ by 1$\%$ (see Table \ref{['tab_loss_func']}), while the dynamical differences of the online predictions are substantial.
• Figure 4: Panel A: Attractor in the HLF case. Panel B: The sojourn episodes within one particular lobe are marked by different colours. Here, the parameters are those used in Lorenz's original paper Lorenz80 except $F_1=0.3027$ in Eq. \ref{['Eq_L9D']}. Panel C: Lobe sojourn time distributions. The exponential fit is calculated over 500 yr-long simulation of Eq. \ref{['Eq_L9D']} and is shown by the black curve $f(t)=a e^{b t}$ with $a=2292$ and $b=-6.05\times 10^{-2}$ with $t$ in day. The inset in panel C shows a magnification of the distribution for the rare and large sojourn times. Panels E and F: Same as panels B and C except that $F_1=6.97\times 10^{-2}$, corresponding to the slow chaotic regime shown in panel D in which the solutions are void of fast oscillations. In this regime, no rare event statistics emerge.
• Figure 5: The L80 attractor vs. its NN-closure in the slow chaos regime. Here $F_1=6.97\times 10^{-2}$ in the L80 model, which corresponds to the slow chaos case shown in Figure \ref{['Fig_sojourn']}D and in CLM16_Lorenz9D.