Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SGNO: Spectral Generator Neural Operators for Stable Long Horizon PDE Rollouts

Jiayi Li, Zhaonan Wang, Flora D. Salim

TL;DR

The Spectral Generator Neural Operator (SGNO), a residual time stepper that targets both linear and nonlinear dynamics, and achieves lower long horizon error and longer stable rollout lengths than strong neural operator baselines on APEBench spanning 1D, 2D, and 3D PDE families.

Abstract

Neural operators provide fast PDE surrogates and often generalize across parameters and resolutions. However, in the short train long test setting, autoregressive rollouts can become unstable. This typically happens for two reasons: one step errors accumulate over time, and high frequency components feed back and grow. We introduce the Spectral Generator Neural Operator (SGNO), a residual time stepper that targets both effects. For the linear part, SGNO uses an exponential time differencing update in Fourier space with a learned diagonal generator. We constrain the real part of this generator to be nonpositive, so iterating the step does not amplify the linear dynamics. For nonlinear dynamics, SGNO adds a gated forcing term with channel mixing within each Fourier mode, which keeps the nonlinear update controlled. To further limit high frequency feedback, SGNO applies spectral truncation and an optional smooth mask on the forcing pathway. We derive a one step amplification bound and a finite horizon rollout error bound. The bound separates generator approximation error from nonlinear mismatch and gives sufficient conditions under which the latent $L^2$ norm does not grow across rollout steps. On APEBench spanning 1D, 2D, and 3D PDE families, SGNO achieves lower long horizon error and longer stable rollout lengths than strong neural operator baselines. Ablations confirm the roles of the generator constraint, gating, and filtering.The code is available at https://github.com/lijy32123-cloud/SGNO.

SGNO: Spectral Generator Neural Operators for Stable Long Horizon PDE Rollouts

TL;DR

The Spectral Generator Neural Operator (SGNO), a residual time stepper that targets both linear and nonlinear dynamics, and achieves lower long horizon error and longer stable rollout lengths than strong neural operator baselines on APEBench spanning 1D, 2D, and 3D PDE families.

Abstract

Neural operators provide fast PDE surrogates and often generalize across parameters and resolutions. However, in the short train long test setting, autoregressive rollouts can become unstable. This typically happens for two reasons: one step errors accumulate over time, and high frequency components feed back and grow. We introduce the Spectral Generator Neural Operator (SGNO), a residual time stepper that targets both effects. For the linear part, SGNO uses an exponential time differencing update in Fourier space with a learned diagonal generator. We constrain the real part of this generator to be nonpositive, so iterating the step does not amplify the linear dynamics. For nonlinear dynamics, SGNO adds a gated forcing term with channel mixing within each Fourier mode, which keeps the nonlinear update controlled. To further limit high frequency feedback, SGNO applies spectral truncation and an optional smooth mask on the forcing pathway. We derive a one step amplification bound and a finite horizon rollout error bound. The bound separates generator approximation error from nonlinear mismatch and gives sufficient conditions under which the latent norm does not grow across rollout steps. On APEBench spanning 1D, 2D, and 3D PDE families, SGNO achieves lower long horizon error and longer stable rollout lengths than strong neural operator baselines. Ablations confirm the roles of the generator constraint, gating, and filtering.The code is available at https://github.com/lijy32123-cloud/SGNO.
Paper Structure (44 sections, 2 theorems, 46 equations, 3 figures, 5 tables)

This paper contains 44 sections, 2 theorems, 46 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. Problem Setup
  5. Training objective.
  6. Evaluation metric.
  7. Aggregate metric.
  8. Method: Spectral Generator Neural Operator
  9. Architecture overview
  10. Pointwise components.
  11. Spectral ETD time advance block
  12. Mode set and truncation.
  13. Stabilized diagonal spectral generator.
  14. ETD update with residual injection.
  15. Implementation notes
  16. ...and 29 more sections

Key Result

Theorem 5.4

Under Assumptions assump:spectral_bound--assump:nonexpansive_trunc_mask, the block map $\Psi_\theta(\cdot;\delta t)$ satisfies where and $L_\sigma$ is the Lipschitz constant of $\sigma$.

Figures (3)

  • Figure 1: (a) Overall architecture of SGNO. The input state is lifted to a latent feature space, propagated through stacked time advance blocks, and projected back to obtain the next state. (b) Time advance block. Features are propagated by a stabilized spectral ETD operator in the Fourier domain, with a nonlinear residual injected through a $\phi_1$ weighted forcing term, followed by a pointwise correction.
  • Figure 2: Long-horizon stability on 1D KdV. Top: nRMSE trajectories (first 100 steps shown for readability); both baseline and SGNO show the per-step median across test trajectories with a $p10$--$p90$ band. Bottom: empirical CDF of per-trajectory stable steps computed on full $T_{\mathrm{eval}}=200$ rollouts with $\tau=0.2$.
  • Figure 3: Long-horizon error trajectories on 1D Dispersion (first 100 steps shown). Under $\tau=0.2$, both methods are largely stable; the main difference is error suppression, where SGNO maintains consistently lower nRMSE throughout the horizon.

Theorems & Definitions (2)

  • Theorem 5.4: One step amplification
  • Corollary 5.5: Nonexpansiveness and gain control