Causally constrained reduced-order neural models of complex turbulent dynamical systems

Abstract

We introduce a flexible framework based on response theory and score matching to suppress spurious, noncausal dependencies in reduced-order neural emulators of turbulent systems, focusing on climate dynamics as a proof-of-concept. We showcase the approach using the stochastic Charney-DeVore model as a relevant prototype for low-frequency atmospheric variability. We show that the resulting causal constraints enhance neural emulators' ability to respond to both weak and strong external forcings, despite being trained exclusively on unforced data. The approach is broadly applicable to modeling complex turbulent dynamical systems in reduced spaces and can be readily integrated into general neural network architectures.

Figures (12)

• Figure 1: Causal constraining of the CdV model (Eq. \ref{['eq:CdV']}) from data through the FDT. Panel (a): The 30 off-diagonal responses $\ln|R_{k,j}(dt)|$ (evaluated at the shortest time scale $t=dt$), flattened and plotted against a dummy index. Panel (b): Unsupervised identification of a response threshold via $k$-means clustering ($k=2$): clusters are colored in black and blue. Panel (c): Causal adjacency matrix: $A_{k,j} = 1$ is assigned if its corresponding response value lies in the higher centroid cluster; otherwise $A_{k,j} = 0$. Panel (d): "causal" deterministic functional form to be enforced in the emulator loss function in Eq. \ref{['eq:causal_loss']}.
• Figure 1: Schematic of the proposed framework. Step 1: From a long trajectory of a Markovian system, the Fluctuation–Dissipation Theorem is used to infer the system’s causal structure. Step 2: This inferred structure is then imposed as a soft constraint during neural-network training, yielding causally constrained emulators. The example shows time series from the Charney–DeVore model (Eq. (3) in the main text) and the discovered causal graph, but the approach applies to general stochastic dynamical systems.
• Figure 2: Perturbed statistics: linear regime. Self-response operator of mean and variance for the variable $x_1$. Panel (a): Time-dependent, response in ensemble mean of variable $x_1(t)$ to an impulse perturbation $\delta x_1(0)$ imposed on $x_1(0)$ at time $t = 0$. Formally: $R_{1,1}(t) = \delta \langle x_1(t) \rangle/\delta x_1(0)$. Panel (b) same as Panel (a) but for response in ensemble variance. Responses are computed using an ensemble of $N_e = 10^6$ members. The magnitude of the impulse perturbation is set to $10^{-1}\sigma_1$, $\sigma_1$ being the standard deviation of $x_1$. Labels: "Numerical" (numerical model), Model vanilla’’ (vanilla, unconstrained, emulator), and "Model causal’’ (causally constrained emulator).
• Figure 2: Time-dependent, responses in ensemble mean $x_k(t)$ to an impulse perturbation in the CdV model (Eq. (3) in the main text) as predicted by the FDT using the quasi-Gaussian and score-matching approximations. Example: column (1), row (2) quantifies the time-dependent mean response of $x_{k = 2}(t)$ given a small, impulse perturbation imposed on $x_{j = 1}(0)$ at time $t = 0$.
• Figure 3: Perturbed statistics: nonlinear regime. Time-dependent response of the ensemble mean and variance of $x_1$ to a strong step function forcing. Forcing: step function $\mathbf{F} = (\sigma_1,0,0,0,0,0)$ for $t \ge 0$, applied on the right-hand side of the numerical model and emulators. $\sigma_1$ is the standard deviation of $x_1$. Responses are computed using an ensemble of $N_e = 10^5$ members. Panel (a): response in ensemble mean. Panel (b): response in ensemble variance. Labels: "Numerical" (numerical model), "Model vanilla" (vanilla, unconstrained, emulator), and "Model causal" (causally constrained emulator).