Spectral Neural Operators

TL;DR

Spectral Neural Operators (SNO) address limitations of sampling-based neural operators by representing both input and output functions with fixed Chebyshev or Fourier series, enabling transparent outputs and eliminating aliasing at fixed resolutions $2N+1$ and $N+1$. The approach analyzes aliasing effects of nonlinear activations, defines a robust super-resolution test, and introduces networks that map coefficient vectors via low-rank operators across multiple basis choices. Across a suite of 1D/2D operators—including integration, parametric ODEs, elliptic, Burgers, KdV, and nonlinear Schrödinger equations—SNO often matches or exceeds the performance of Fourier Neural Operators and DeepONet, particularly for smooth problems, while highlighting limitations tied to Gibbs phenomena and fixed bases. The work provides a practical, spectrally-grounded alternative to neural operators with improved interpretability and stability, and outlines concrete paths for future enhancements such as adaptive bases and Gibbs-robust reconstructions.

Abstract

A plentitude of applications in scientific computing requires the approximation of mappings between Banach spaces. Recently introduced Fourier Neural Operator (FNO) and Deep Operator Network (DeepONet) can provide this functionality. For both of these neural operators, the input function is sampled on a given grid (uniform for FNO), and the output function is parametrized by a neural network. We argue that this parametrization leads to 1) opaque output that is hard to analyze and 2) systematic bias caused by aliasing errors in the case of FNO. The alternative, advocated in this article, is to use Chebyshev and Fourier series for both domain and codomain. The resulting Spectral Neural Operator (SNO) has transparent output, never suffers from aliasing, and may include many exact (lossless) operations on functions. The functionality is based on well-developed fast, and stable algorithms from spectral methods. The implementation requires only standard numerical linear algebra. Our benchmarks show that for many operators, SNO is superior to FNO and DeepONet.

Figures (3)

• Figure 1: (a) the output of neural network $N(x)$ computed on coarse and fine grids. On each subgrid, loss and gradients are zero, so the network provides the best (alas, pathological) approximation to $f(x) = 2x$ on the interval $[-1, 1]$ for all but the finest grid. (b) FNO trained to take the first derivative of random trigonometric polynomials on grid $2h$ is evaluated for the same functions on grid $h$. The difference between outputs for a particular function is on the right; statistics for $750$ functions is on the left. The average discrepancy is about $25\%$ of $L_2$ norm. Inconsistency of FNO results from aliasing of high harmonics produced by activation functions.
• Figure 2: (a) On the left, relative aliasing error for FNO $N(u, x)$, computed as $\left\|N(u(x_{2h}), x_{2h}) - \left[N(u(x_{h}), x_{h})\right]_{2h}\right\|$ normalized on $\left\|\left[N(u(x_{h}), x_{h})\right]_{2h}\right\|$, where $\left[\cdot\right]_{2h}$ is a projection on grid with spacing $2h$. FNO was trained on the grid with $n$ point ($x$ axis) on functions with frequencies $[0, 10]$. Aliasing error decreases when larger grids are used. On the right, relative test error for the input with $n$ point ($x$ axis) for FNO trained for input with $n=21$ points. Relative test errors is larger for finer grid, which means FNO has systematic bias caused by aliasing. (b) Neural operators were trained on input with harmonics $[k_{\min}, k_{\max}]$ and grid with $100$ points and evaluated on input with harmonics $[k_{\min} + \Delta k, k_{\max} + \Delta k]$ on grid with $300$ points. Relative test error sharply increases with $\Delta k$, super-resolution is not observed.
• Figure 3: Linear algebra for Chebyshev polynomials.

Theorems & Definitions (2)

• Theorem 1: Aliasing and ${\sf ReLU}$
• proof