Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PODNO: Proper Orthogonal Decomposition Neural Operators

Zilan Cheng, Zhongjian Wang, Li-Lian Wang, Mejdi Azaiez

TL;DR

This paper tackles the challenge of solving PDEs with strong high-frequency content by introducing PODNO, a POD-based neural operator that replaces the Fourier kernel with a data-driven POD transform within a Generalized Spectral Operator framework. It develops a universal approximation theory for GSO, provides a concrete PODNO algorithm, and shows via experiments on Darcy, NLS, and KP equations that PODNO can outperform Fourier-based approaches in accuracy and efficiency for dispersive, high-frequency problems. The work also compares PODNO to POD-accelerated splitting methods, offering insights into when transform choices and mode counts matter most, and includes thorough ablations to map out parameter sensitivities. Overall, PODNO offers a robust, non-periodic, energy-efficient alternative for learning operators on high-frequency PDEs with potential broad impact on dispersive simulations and complex geometries. The combination of theory (GSO universality) and empirical evidence (NLS, KP, and Darcy) underlines PODNO’s potential as a versatile tool for mesh-invariant, high-frequency operator learning.

Abstract

In this paper, we introduce Proper Orthogonal Decomposition Neural Operators (PODNO) for solving partial differential equations (PDEs) dominated by high-frequency components. Building on the structure of Fourier Neural Operators (FNO), PODNO replaces the Fourier transform with (inverse) orthonormal transforms derived from the Proper Orthogonal Decomposition (POD) method to construct the integral kernel. Due to the optimality of POD basis, the PODNO has potential to outperform FNO in both accuracy and computational efficiency for high-frequency problems. From analysis point of view, we established the universality of a generalization of PODNO, termed as Generalized Spectral Operator (GSO). In addition, we evaluate PODNO's performance numerically on dispersive equations such as the Nonlinear Schrodinger (NLS) equation and the Kadomtsev-Petviashvili (KP) equation.

PODNO: Proper Orthogonal Decomposition Neural Operators

TL;DR

This paper tackles the challenge of solving PDEs with strong high-frequency content by introducing PODNO, a POD-based neural operator that replaces the Fourier kernel with a data-driven POD transform within a Generalized Spectral Operator framework. It develops a universal approximation theory for GSO, provides a concrete PODNO algorithm, and shows via experiments on Darcy, NLS, and KP equations that PODNO can outperform Fourier-based approaches in accuracy and efficiency for dispersive, high-frequency problems. The work also compares PODNO to POD-accelerated splitting methods, offering insights into when transform choices and mode counts matter most, and includes thorough ablations to map out parameter sensitivities. Overall, PODNO offers a robust, non-periodic, energy-efficient alternative for learning operators on high-frequency PDEs with potential broad impact on dispersive simulations and complex geometries. The combination of theory (GSO universality) and empirical evidence (NLS, KP, and Darcy) underlines PODNO’s potential as a versatile tool for mesh-invariant, high-frequency operator learning.

Abstract

In this paper, we introduce Proper Orthogonal Decomposition Neural Operators (PODNO) for solving partial differential equations (PDEs) dominated by high-frequency components. Building on the structure of Fourier Neural Operators (FNO), PODNO replaces the Fourier transform with (inverse) orthonormal transforms derived from the Proper Orthogonal Decomposition (POD) method to construct the integral kernel. Due to the optimality of POD basis, the PODNO has potential to outperform FNO in both accuracy and computational efficiency for high-frequency problems. From analysis point of view, we established the universality of a generalization of PODNO, termed as Generalized Spectral Operator (GSO). In addition, we evaluate PODNO's performance numerically on dispersive equations such as the Nonlinear Schrodinger (NLS) equation and the Kadomtsev-Petviashvili (KP) equation.

Paper Structure

This paper contains 30 sections, 6 theorems, 100 equations, 18 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Neural Operators
  3. General setup in operator learning
  4. FNO structure
  5. Generalized Spectral Operators and PODNO
  6. Generalized spectral operators
  7. POD basis generation and POD layers
  8. PODNO algorithm
  9. Universal approximation and error bounds for GSO
  10. Numerical Results
  11. Setting and error metrics
  12. Validation on the Darcy problem
  13. Nonlinear Schrödinger equations
  14. Kadomtsev-Petviashvili (KP) equation
  15. Comparison of PODNO with time-splitting spectral methods accelerated by a POD basis
  16. ...and 15 more sections

Key Result

Theorem 3.1

Let $\Omega\subset\mathbb{R}^d$, and let $\mathcal{G}: H^s(\Omega; \mathbb{R}^{d_a})\rightarrow H^{s'}(\Omega; \mathbb{R}^{d_u})$ be a continuous operator, where $s'\geq s>d/2$. Suppose $K \subset H^s(\Omega; \mathbb{R}^{d_a})$ is a compact subset. Then, $\forall~\varepsilon>0$, $\exists$ a GSO, $\m

Figures (18)

  • Figure 2.1: FNO. (a) The architecture of FNO consists of 3 parts: dimension lifting $\mathcal{P}$; a sequence of Fourier layers each followed by a nonlinear activation function except the final one; and dimension reduction $\mathcal{Q}$. (b) Each Fourier layer is given by $\mathscr{F}_N^{-1}[\mathcal{R}_l\circ\mathscr{F}_N[v_{(l)}]]+\mathcal{W}_l[v_{(l)}]$.
  • Figure 3.1: Generalized Spectral Operators (GSO). (a) The architecture of GSO comprises three main components: a dimension lifting $\mathcal{P}$; a sequence of GSO layers $\mathcal{L}_l$ that incorporate physical parameters, and apply a nonlinear activation function $\sigma$ except in the final layer; and a dimension reduction $\mathcal{Q}$. (b) The GSO layers $\mathcal{L}_l=\varPi_N^{-1}[\mathcal{R}_l\cdot\varPi_N [v_{(l)}]]+\mathcal{W}_l[v_{(l)}]$ . (c) The POD basis for a family of KP equations, which will be explicitly described in Subsection \ref{['kp results']}, provides an orthonormal set that can serve as the basis for the GSO layers.
  • Figure 4.1: Visualization of the Darcy problem: (Top) Four realizations of random permeability fields $a$ generated according to \ref{['darcy_initial_choice']}; (Middle) the corresponding ground truth solutions; (Bottom) the PODNO predictions using the same input fields as in the top row.
  • Figure 4.2: Test errors comparison between FNO and PODNO for the Darcy problem
  • Figure 4.3: Gaussian-type wave packet initial conditions for the NLS equation, with four randomly generated instances. The top row displays the real part, while the bottom row shows the corresponding imaginary part.
  • ...and 13 more figures

Theorems & Definitions (14)

  • Remark 2.1
  • Definition 3.1: GSO
  • Theorem 3.1: Universal approximation theorem
  • Remark 3.1
  • Definition A.1
  • Definition A.2
  • Proposition A.1: Sobolev embedding theorem evans2022partial
  • Proposition A.2: Sobolev product estimate kato1988commutator
  • Proposition A.3: Universal approximation theorem for ordinary neural networks hornik1991approximation
  • Lemma A.1
  • ...and 4 more