SVD-NO: Learning PDE Solution Operators with SVD Integral Kernels

TL;DR

SVD-NO introduces a neural operator that explicitly parameterizes the kernel via singular value decomposition, learning left and right singular functions and singular values with an orthogonality penalty to enforce a low-rank, expressive kernel. By factorizing the kernel and applying it in the low-rank basis, the method achieves expressive dependence on the input function and long-range interactions while maintaining scalable, linear-time per-layer complexity. Across five diverse PDE benchmarks, SVD-NO attains state-of-the-art accuracy, with larger gains on highly spatially variable solutions, and ablation studies confirm the value of the SVD parameterization and the orthogonality constraint. The approach offers a principled bridge between operator theory and neural methods, and its publicly available code enables practical deployment and further research in neural PDE solvers.

Abstract

Neural operators have emerged as a promising paradigm for learning solution operators of partial differential equa- tions (PDEs) directly from data. Existing methods, such as those based on Fourier or graph techniques, make strong as- sumptions about the structure of the kernel integral opera- tor, assumptions which may limit expressivity. We present SVD-NO, a neural operator that explicitly parameterizes the kernel by its singular-value decomposition (SVD) and then carries out the integral directly in the low-rank basis. Two lightweight networks learn the left and right singular func- tions, a diagonal parameter matrix learns the singular values, and a Gram-matrix regularizer enforces orthonormality. As SVD-NO approximates the full kernel, it obtains a high de- gree of expressivity. Furthermore, due to its low-rank struc- ture the computational complexity of applying the operator remains reasonable, leading to a practical system. In exten- sive evaluations on five diverse benchmark equations, SVD- NO achieves a new state of the art. In particular, SVD-NO provides greater performance gains on PDEs whose solutions are highly spatially variable. The code of this work is publicly available at https://github.com/2noamk/SVDNO.git.

Figures (3)

• Figure 1: SVD-NO. The coefficient–coordinate tuple $z=(a(x),x)$ is fed through two singular function nets, $\mathrm{NN}_{\!\Phi}$ and $\mathrm{NN}_{\!\Psi}$, producing global basis functions $\Phi(z)$ and $\Psi(z)$. An encoder $P$ maps $z$ to the initial latent state, $v^{0}$, which is then processed by $T$SVD blocks to reach $v^{T}$. Each block applies a kernel $K(v)=\Psi(z)\,\Sigma\!\int\!\Phi(z')\,v(z')\,dz'$ (with trainable diagonal $\Sigma$), adds a point-wise map $Wv$, followed by activation $\sigma$. A decoder $Q$ maps the final latent state, $v^T$, to the predicted solution $u$. Dashed arrows indicate loss paths: Gram matrices $G_{\!\Phi},G_{\!\Psi}$ drive the orthogonality penalty $L_{\text{orth}}$, with $u$ supervising $L_{2}(y,u)$.
• Figure 2: SVD-NO’s percentage improvement over the best baseline versus the solution's spatial variability ($\beta$) for each PDE. Each point represents a dataset; the least‐squares fit shows that more challenging problems (higher solution spatial variability) correlate with larger improvements.
• Figure 3: (Left) Min–max normalized test‐loss curves and (Right) min–max normalized peak GPU memory usage vs. SVD rank $L$. Increasing $L$ yields significant accuracy gains but also linearly higher memory, illustrating the practical trade‐off between model fidelity and resource consumption.