Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Operator Learning: Algorithms and Analysis

Nikola B. Kovachki, Samuel Lanthaler, Andrew M. Stuart

TL;DR

Operator learning seeks to approximate maps between infinite-dimensional function spaces, with neural operators as practical surrogates for PDE-based models. The survey surveys three main architectures—PCA-Net, DeepONet, and Fourier Neural Operator (FNO)—and a random-features variant, outlining universal approximation results and transitioning to quantitative error and complexity analyses. It highlights a tension: while universal approximation holds broadly, standard architectures face a curse of parametric complexity for general Lipschitz operators, whereas holomorphic and structured operators admit algebraic or better rates, especially when emulating numerical methods or exploiting Barron-type structure. The work thus maps when neural operators can efficiently surrogate complex PDE operators, and under what structural assumptions one can guarantee scalable, discretization-invariant performance.

Abstract

Operator learning refers to the application of ideas from machine learning to approximate (typically nonlinear) operators mapping between Banach spaces of functions. Such operators often arise from physical models expressed in terms of partial differential equations (PDEs). In this context, such approximate operators hold great potential as efficient surrogate models to complement traditional numerical methods in many-query tasks. Being data-driven, they also enable model discovery when a mathematical description in terms of a PDE is not available. This review focuses primarily on neural operators, built on the success of deep neural networks in the approximation of functions defined on finite dimensional Euclidean spaces. Empirically, neural operators have shown success in a variety of applications, but our theoretical understanding remains incomplete. This review article summarizes recent progress and the current state of our theoretical understanding of neural operators, focusing on an approximation theoretic point of view.

Operator Learning: Algorithms and Analysis

TL;DR

Operator learning seeks to approximate maps between infinite-dimensional function spaces, with neural operators as practical surrogates for PDE-based models. The survey surveys three main architectures—PCA-Net, DeepONet, and Fourier Neural Operator (FNO)—and a random-features variant, outlining universal approximation results and transitioning to quantitative error and complexity analyses. It highlights a tension: while universal approximation holds broadly, standard architectures face a curse of parametric complexity for general Lipschitz operators, whereas holomorphic and structured operators admit algebraic or better rates, especially when emulating numerical methods or exploiting Barron-type structure. The work thus maps when neural operators can efficiently surrogate complex PDE operators, and under what structural assumptions one can guarantee scalable, discretization-invariant performance.

Abstract

Operator learning refers to the application of ideas from machine learning to approximate (typically nonlinear) operators mapping between Banach spaces of functions. Such operators often arise from physical models expressed in terms of partial differential equations (PDEs). In this context, such approximate operators hold great potential as efficient surrogate models to complement traditional numerical methods in many-query tasks. Being data-driven, they also enable model discovery when a mathematical description in terms of a PDE is not available. This review focuses primarily on neural operators, built on the success of deep neural networks in the approximation of functions defined on finite dimensional Euclidean spaces. Empirically, neural operators have shown success in a variety of applications, but our theoretical understanding remains incomplete. This review article summarizes recent progress and the current state of our theoretical understanding of neural operators, focusing on an approximation theoretic point of view.
Paper Structure (57 sections, 8 theorems, 88 equations, 5 figures, 1 table)

This paper contains 57 sections, 8 theorems, 88 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. High Dimensional Vectors Versus Functions
  3. Literature Review
  4. Algorithms on Function Space
  5. Supervised Learning on Function Space
  6. Approximation Theory
  7. Overview of Paper
  8. Operator Learning
  9. Supervised Learning
  10. Supervised Learning of Operators
  11. Training and Testing
  12. Finding Latent Structure
  13. Example (Fluid Flow in a Porous Medium)
  14. Specific Supervised Learning Architectures
  15. PCA-Net
  16. ...and 42 more sections

Key Result

theorem 4.1

Suppose that $\mathcal{U}$, $\mathcal{V}$ are separable Banach spaces with the approximation property. Let $\Psi^\dagger : \mathcal{U} \to \mathcal{V}$ be a continuous operator. Fix a compact set $K \subset \mathcal{U}$. Then for any $\epsilon > 0$, there exist positive integers $d_{\mathcal{U}},d_{

Figures (5)

  • Figure 1: Latent Structure in Maps Between Banach Spaces $\mathcal{U}$ and $\mathcal{V}$
  • Figure 2: Special case of an averaging neural operator, illustrated as a nonlinear encoder-decoder architecture; with encoder $u\mapsto \alpha = \int_{\mathfrak{D}} \phi(u(y),y) \, dy$, and decoder $\alpha \mapsto \psi(\alpha,{\,\cdot\,})$.
  • Figure :
  • Figure :
  • Figure :

Theorems & Definitions (11)

  • theorem 4.1
  • theorem 4.2
  • remark 4.3
  • theorem 4.4: PCA-Net universality
  • theorem 4.5
  • corollary 4.6
  • theorem 5.1
  • theorem 5.2
  • remark 5.3
  • proposition 5.4: Curse of Parametric Complexity
  • ...and 1 more