Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SpectraKAN: Conditioning Spectral Operators

Chun-Wun Cheng, Carola-Bibiane Schönlieb, Angelica I. Aviles-Rivero

TL;DR

SpectraKAN tackles the limitation of static Fourier kernels in neural operators by introducing an input-conditioned spectral operator. It combines Adaptive Multi-Scale FNO (AMFNO) with a Global Conditioning Kolmogorov–Arnol'd Network (G-KAN) and a Global Modulation Layer (GML) to produce a context-aware operator: $G(u)(x)=f(u)(x)+A_{\,\Phi_{KAN}(u)}[f(u)](x)$. Theoretical results show the attention-based modulation converges to a resolution-independent continuous operator with Lipschitz-controlled global modulation, and experiments across diverse PDE benchmarks achieve state-of-the-art performance, especially on challenging spatio-temporal tasks. This framework offers a principled, scalable way to adapt spectral kernels to global states, improving accuracy, stability, and interpretability in PDE forecasting.

Abstract

Spectral neural operators, particularly Fourier Neural Operators (FNO), are a powerful framework for learning solution operators of partial differential equations (PDEs) due to their efficient global mixing in the frequency domain. However, existing spectral operators rely on static Fourier kernels applied uniformly across inputs, limiting their ability to capture multi-scale, regime-dependent, and anisotropic dynamics governed by the global state of the system. We introduce SpectraKAN, a neural operator that conditions the spectral operator on the input itself, turning static spectral convolution into an input-conditioned integral operator. This is achieved by extracting a compact global representation from spatio-temporal history and using it to modulate a multi-scale Fourier trunk via single-query cross-attention, enabling the operator to adapt its behaviour while retaining the efficiency of spectral mixing. We provide theoretical justification showing that this modulation converges to a resolution-independent continuous operator under mesh refinement and KAN gives smooth, Lipschitz-controlled global modulation. Across diverse PDE benchmarks, SpectraKAN achieves state-of-the-art performance, reducing RMSE by up to 49% over strong baselines, with particularly large gains on challenging spatio-temporal prediction tasks.

SpectraKAN: Conditioning Spectral Operators

TL;DR

SpectraKAN tackles the limitation of static Fourier kernels in neural operators by introducing an input-conditioned spectral operator. It combines Adaptive Multi-Scale FNO (AMFNO) with a Global Conditioning Kolmogorov–Arnol'd Network (G-KAN) and a Global Modulation Layer (GML) to produce a context-aware operator: . Theoretical results show the attention-based modulation converges to a resolution-independent continuous operator with Lipschitz-controlled global modulation, and experiments across diverse PDE benchmarks achieve state-of-the-art performance, especially on challenging spatio-temporal tasks. This framework offers a principled, scalable way to adapt spectral kernels to global states, improving accuracy, stability, and interpretability in PDE forecasting.

Abstract

Spectral neural operators, particularly Fourier Neural Operators (FNO), are a powerful framework for learning solution operators of partial differential equations (PDEs) due to their efficient global mixing in the frequency domain. However, existing spectral operators rely on static Fourier kernels applied uniformly across inputs, limiting their ability to capture multi-scale, regime-dependent, and anisotropic dynamics governed by the global state of the system. We introduce SpectraKAN, a neural operator that conditions the spectral operator on the input itself, turning static spectral convolution into an input-conditioned integral operator. This is achieved by extracting a compact global representation from spatio-temporal history and using it to modulate a multi-scale Fourier trunk via single-query cross-attention, enabling the operator to adapt its behaviour while retaining the efficiency of spectral mixing. We provide theoretical justification showing that this modulation converges to a resolution-independent continuous operator under mesh refinement and KAN gives smooth, Lipschitz-controlled global modulation. Across diverse PDE benchmarks, SpectraKAN achieves state-of-the-art performance, reducing RMSE by up to 49% over strong baselines, with particularly large gains on challenging spatio-temporal prediction tasks.
Paper Structure (32 sections, 7 theorems, 91 equations, 8 figures, 11 tables)

This paper contains 32 sections, 7 theorems, 91 equations, 8 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Deep learning for PDEs
  4. Methodology
  5. Architecture Overview
  6. Adaptive Multi-Scale FNO (AMFNO)
  7. Spectral convolution with gated axial spectra.
  8. Fine-scale operator.
  9. Coarse branch and fusion.
  10. Global Conditioning KAN
  11. KAN-based Global Token Conditioning.
  12. Global context extraction.
  13. Kolmogorov--Arnold Network (KAN) encoder.
  14. Spline-based edge parametrization.
  15. Approximation and interpretability.
  16. ...and 17 more sections

Key Result

Lemma 3.1

Consider an edge function of the form $\phi(x) = w_b\, b(x) + w_s \sum_{r} c_r\, B_r^{(k)}(x;\Xi)$, where $b \in C^1$, and $B_r^{(k)}$ are order-$k$ B-splines defined on a knot grid $\Xi = \{t_j\}$, with minimum knot spacing $h_{\min} := \min_j (t_{j+1} - t_j) > 0$. Assume that $\sup_x |b'(x)| \le L

Figures (8)

  • Figure 1: Overview of our SpectraKAN architecture. (a) AFNO performs spectral mixing via FFT–gating–IFFT with Squeeze–Excitation. (b) KAN models nonlinear interactions with learnable spline functions. (c) A global-conditioned KAN produces a global token (query) that fuses with multi-scale AFNO features (key/value), followed by a lightweight projection to obtain the prediction.
  • Figure 2: Top: 1D compressible Navier–Stokes solution evolution (space–time map). Bottom: 2D Darcy flow solution snapshot. While several baselines recover the coarse structure, they often suffer from smoothing or artifacts; our method produces predictions that most closely resemble the ground truth across both settings.
  • Figure 3: Qualitative comparison on the Shallow Water benchmark. Shown are the initial conditions, the predicted and ground-truth solutions, and the absolute error at t=0.16s.
  • Figure 4: Qualitative comparison across Local FNO and ours in 2D shallow water.
  • Figure 5: Full-resolution comparison on the climate modeling benchmark. Shown are the input temperature field, the model prediction, the ground truth, and the absolute error with respect to the reference solution..
  • ...and 3 more figures

Theorems & Definitions (11)

  • Lemma 3.1: Edge Lipschitz bound for spline KAN
  • Theorem 3.2: Smooth, Lipschitz-controlled global modulation
  • Proposition 3.3
  • Lemma 1.1: Edge Lipschitz bound for spline KAN
  • proof
  • Theorem 1.2
  • proof
  • Proposition 1.3
  • proof
  • Theorem 1.4
  • ...and 1 more