SFO: Learning PDE Operators via Spectral Filtering

TL;DR

SFO tackles the challenge of learning PDE solution operators with long-range dependencies by fixing a global spectral basis, the Universal Spectral Basis (USB), derived from Hilbert matrix eigenvectors, and learning only a compact set of spectral coefficients. Grounded in the observed linear dynamical systems structure of discrete Green’s functions for shift-invariant discretizations, SFO achieves efficient, high-fidelity kernel representations via a truncated USB expansion and FFT-based convolution within Spectral Transform Units. Theoretical results show that for stable local discretizations, inverse operators admit approximation with $L = \tilde{O}(\log(1/\\epsilon))$ USB modes, while empirical results across six PDE benchmarks demonstrate state-of-the-art accuracy with substantially fewer parameters than baselines. The work highlights the practical impact of a fixed global basis in neural operators, enabling scalable, long-range PDE modeling, and points to future directions including boundary-aware and nonuniform-grid extensions. $L = \tilde{O}(\log(1/\\epsilon))$ is a key theoretical takeaway, underpinning the efficiency of the USB truncation in SFO.

Abstract

Partial differential equations (PDEs) govern complex systems, yet neural operators often struggle to efficiently capture the long-range, nonlocal interactions inherent in their solution maps. We introduce Spectral Filtering Operator (SFO), a neural operator that parameterizes integral kernels using the Universal Spectral Basis (USB), a fixed, global orthonormal basis derived from the eigenmodes of the Hilbert matrix in spectral filtering theory. Motivated by our theoretical finding that the discrete Green's functions of shift-invariant PDE discretizations exhibit spatial Linear Dynamical System (LDS) structure, we prove that these kernels admit compact approximations in the USB. By learning only the spectral coefficients of rapidly decaying eigenvalues, SFO achieves a highly efficient representation. Across six benchmarks, including reaction-diffusion, fluid dynamics, and 3D electromagnetics, SFO achieves state-of-the-art accuracy, reducing error by up to 40% relative to strong baselines while using substantially fewer parameters.

Figures (5)

• Figure 1: SFO architecture. The input field $(a(x), x)$, is lifted by a map $P$ to a latent representation $v^0$. The latent field is updated by $T$ stacked STU layers, each applying an operator $\mathcal{K}$. Finally, a projection $Q$ maps the last latent state $v^T$ back to the solution field $u(x)$.
• Figure 2: STU layer: the current state $v^t$ is normalized ($N$), passed through the operator $\mathcal{K}$, then through a linear transformation with a nonlinearity, and added back to $v^t$. The operator $\mathcal{K}$ itself is implemented via STU in the Fourier domain using the top $L$ Hilbert eigenmodes $\{\phi_l\}_{l=1}^L$, $\mathcal{K} v^t = \sum_{l=1}^L \Theta_l\,\mathcal{F}^{-1}(\widehat{v^t}\odot\widehat{\phi_l})$.
• Figure 3: Magnitude of learned Hilbert spectral coefficients vs. mode index $l$. We plot the Frobenius norms $\|\Theta_ l\|_F$ averaged across STU layers. Coefficients decay with $l$, indicating SFO concentrates most of its capacity on a few low-index global modes.
• Figure 4: Test loss vs. STU rank $L$. The test loss decreases rapidly for small $L$, indicating that the leading modes capture most of the structure, and then flattens as additional modes give only marginal gains. All curves are min--max normalized to a common scale.
• Figure 5: Average magnitude of learned Hilbert spectral coefficients across benchmarks. For each STU layer, we compute the Frobenius norm $\|\Theta_ l\|_F$ of the coefficient matrix associated with mode $l$ and average across layers. The learned coefficients emphasize low-index modes, supporting the use of a truncated Hilbert expansion.

Theorems & Definitions (6)

• Theorem 7.1: Learnability of stable local shift-invariant discretizations
• Lemma 7.2: Block stencil yields matrix-valued exponential decay
• proof
• Lemma 7.3: Spectral filtering approximation for exponentially decaying kernels hazan2017spectral
• proof : Proof sketch
• proof : Proof of Theorem \ref{['thm:main_learnability']}