Stabilizing autoregressive forecasts in chaotic systems via multi-rate latent recurrence

TL;DR

MSR-HINE addresses the core difficulty of long-horizon forecasting in chaotic dynamical systems by introducing a multiscale latent recurrence that propagates latent content at multiple temporal scales, coupled with an implicit one-step predictor and a scale-aware posterior fusion. The method stabilizes autoregressive rollouts by maintaining slow-manifold context while allowing fast fluctuations to be conditioned on this context, and by correcting latent trajectories via hidden-state coupling. Across KS and L96 benchmarks, MSR-HINE yields substantial improvements in RMSE, ACC, and spectral-energy fidelity relative to strong U-Net baselines and a two-level HINE variant, extending predictability horizons and preserving multiscale structure. These results suggest practical impact for geophysical surrogates, turbulence modeling, and digital twins, where robust multiscale forecasting is essential. The work points to future directions in data assimilation integration, adaptive timescale scheduling, and application to higher-dimensional PDEs.

Abstract

Long-horizon autoregressive forecasting of chaotic dynamical systems remains challenging due to rapid error amplification and distribution shift: small one-step inaccuracies compound into physically inconsistent rollouts and collapse of large-scale statistics. We introduce MSR-HINE, a hierarchical implicit forecaster that augments multiscale latent priors with multi-rate recurrent modules operating at distinct temporal scales. At each step, coarse-to-fine recurrent states generate latent priors, an implicit one-step predictor refines the state with multiscale latent injections, and a gated fusion with posterior latents enforces scale-consistent updates; a lightweight hidden-state correction further aligns recurrent memories with fused latents. The resulting architecture maintains long-term context on slow manifolds while preserving fast-scale variability, mitigating error accumulation in chaotic rollouts. Across two canonical benchmarks, MSR-HINE yields substantial gains over a U-Net autoregressive baseline: on Kuramoto-Sivashinsky it reduces end-horizon RMSE by 62.8% at H=400 and improves end-horizon ACC by +0.983 (from -0.155 to 0.828), extending the ACC >= 0.5 predictability horizon from 241 to 400 steps; on Lorenz-96 it reduces RMSE by 27.0% at H=100 and improves end horizon ACC by +0.402 (from 0.144 to 0.545), extending the ACC >= 0.5 horizon from 58 to 100 steps.

Figures (11)

• Figure 1: MSR-HINE architecture for one-step forecasting within TBPTT. A length-$L_{\mathrm{tbptt}}$ window from the KS/L96 trajectory is encoded into a hierarchy of multiscale latents $\{z_n^{(l)}\}_{l=1}^L$ using spectral/pooling encoders $T^{(l)}$ (coarser with increasing level). At each level, a recurrent unit (GRU) propagates a multi-rate hidden state $h_{n+1}^{(l)}$ and produces a latent prior $z_{n+1|n}^{(l)}$, which is fused with the hidden state to form scale-matched conditioning inputs for a periodic 1D U-Net. The U-Net predicts the next state $\hat{u}_{n+1}$ while injecting the multiscale priors at corresponding encoder depths. Posterior latents $\{\tilde{z}_{n+1}^{(l)}\}$ are then extracted from $\hat{u}_{n+1}$ via the same hierarchy of encoders and combined with the priors through a gated fusion module $\sigma^{(l)}$ to obtain fused latents $\{z_{n+1}^{(l)}\}$ used for the next step. The procedure is iterated autoregressively to generate long-horizon rollouts.
• Figure 2: Autoregressive KS state profiles (L=1): U-Net-AR vs. MSR-HINE. Representative spatial profiles $u(x)$ at increasing rollout horizons (columns). Solid black: ground truth (GT). Green dashed: U-Net-AR prediction. Blue dashed: MSR-HINE prediction. The "diff" panels report the pointwise error (prediction $-$ GT), highlighting stronger error growth and drift for U-Net-AR at longer horizons, while MSR-HINE remains more stable under self-fed rollouts.
• Figure 3: Autoregressive L96 state profiles (L=1): U-Net-AR vs. MSR-HINE. Representative L96 state vectors $u(x)$ (with $N=40$ cyclic coordinates) shown at horizons $t=\{30,60,90\}$. Solid black: ground truth (GT). Green dashed: U-Net-AR prediction. Blue dashed: MSR-HINE prediction. The "diff" panels show pointwise errors (prediction $-$ GT). Compared to U-Net-AR, MSR-HINE exhibits reduced drift and slower error growth over the rollout, indicating improved stability in long-horizon autoregressive forecasting.
• Figure 4: Kuramoto--Sivashinsky (KS) long-horizon forecast skill under autoregressive rollouts. Left: RMSE versus horizon. Right: anomaly correlation coefficient (ACC) versus horizon, computed as the cosine similarity between predicted and ground-truth anomalies (space-mean removed at each time). Solid lines denote the mean over test rollouts and shaded bands indicate $\pm$1 standard deviation. MSR-HINE exhibits slower error growth and maintains higher anomaly correlation over longer horizons compared with HINE-L2 and the U-Net-AR baseline.
• Figure 5: L96 long-horizon forecast skill under autoregressive rollouts. Left: RMSE versus horizon. Right: anomaly correlation coefficient (ACC) versus horizon, computed as the cosine similarity between predicted and ground-truth anomalies (space-mean removed at each time). Solid lines denote the mean over test rollouts and shaded bands indicate $\pm$1 standard deviation. Consistent with the more strongly chaotic dynamics, all methods decorrelate more rapidly than in KS; however, MSR-HINE sustains lower RMSE and higher ACC across the rollout relative to HINE-L2 and the U-Net-AR baseline.