Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Controlling Statistical, Discretization, and Truncation Errors in Learning Fourier Linear Operators

Unique Subedi, Ambuj Tewari

TL;DR

The paper analyzes learning Fourier linear operators within the operator-learning framework by focusing on the linear Fourier layer of Fourier Neural Operators. It identifies three error sources—statistical, discretization, and truncation—and develops a DFT-based constrained least-squares estimator to quantify them, yielding non-asymptotic excess-risk bounds. The main results establish an upper bound of order $O\left( 1/\sqrt{n} + 1/N^{s} + 1/K^{2s} \right)$ and a matching lower bound with $O\left( 1/n + 1/N^{2s} + 1/K^{2s} \right)$ under Sobolev smoothness $s > d/2$, together with a detailed decomposition of errors and a formal link to FDA concepts. Empirical experiments in low dimensions corroborate the theoretical rates, illustrating the separate contributions of statistical, truncation, and discretization errors and showcasing multiresolution generalization when training at coarser grids. The work provides a rigorous learning-theoretic foundation for operator learning, highlights the role of Fourier-domain truncation and grid discretization, and points to future directions including multi-layer extensions and active-learning strategies for PDE solution operators.

Abstract

We study learning-theoretic foundations of operator learning, using the linear layer of the Fourier Neural Operator architecture as a model problem. First, we identify three main errors that occur during the learning process: statistical error due to finite sample size, truncation error from finite rank approximation of the operator, and discretization error from handling functional data on a finite grid of domain points. Finally, we analyze a Discrete Fourier Transform (DFT) based least squares estimator, establishing both upper and lower bounds on the aforementioned errors.

Controlling Statistical, Discretization, and Truncation Errors in Learning Fourier Linear Operators

TL;DR

The paper analyzes learning Fourier linear operators within the operator-learning framework by focusing on the linear Fourier layer of Fourier Neural Operators. It identifies three error sources—statistical, discretization, and truncation—and develops a DFT-based constrained least-squares estimator to quantify them, yielding non-asymptotic excess-risk bounds. The main results establish an upper bound of order and a matching lower bound with under Sobolev smoothness , together with a detailed decomposition of errors and a formal link to FDA concepts. Empirical experiments in low dimensions corroborate the theoretical rates, illustrating the separate contributions of statistical, truncation, and discretization errors and showcasing multiresolution generalization when training at coarser grids. The work provides a rigorous learning-theoretic foundation for operator learning, highlights the role of Fourier-domain truncation and grid discretization, and points to future directions including multi-layer extensions and active-learning strategies for PDE solution operators.

Abstract

We study learning-theoretic foundations of operator learning, using the linear layer of the Fourier Neural Operator architecture as a model problem. First, we identify three main errors that occur during the learning process: statistical error due to finite sample size, truncation error from finite rank approximation of the operator, and discretization error from handling functional data on a finite grid of domain points. Finally, we analyze a Discrete Fourier Transform (DFT) based least squares estimator, establishing both upper and lower bounds on the aforementioned errors.
Paper Structure (28 sections, 11 theorems, 142 equations, 3 figures)

This paper contains 28 sections, 11 theorems, 142 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Neural Operators
  3. Fourier Neural Operator (FNO)
  4. Our Contribution
  5. Related Works
  6. Preliminaries
  7. Notation
  8. $L^2$-Spaces and Fourier Analysis
  9. Sobolev Spaces
  10. Learning Fourier Linear Operators
  11. Problem Setting and Error Types
  12. Further Connection to FDA.
  13. A Constrained Least-Squares Estimator
  14. Error Bounds
  15. Experiments
  16. ...and 13 more sections

Key Result

Proposition 1

If $\lambda \in \ell^1(\mathbb{Z}^d)$, then where the equality holds for every $u \in L^2(\mathbb{T}^d, \mathbb{R})$.

Figures (3)

  • Figure 1: Statistical error of the estimator.
  • Figure 2: Truncation error of the estimator.
  • Figure 3: Discretization error of the estimator.

Theorems & Definitions (22)

  • Proposition 1
  • Theorem 1: Upper Bound
  • Theorem 2: Lower Bound
  • Proposition 2
  • proof
  • proof
  • proof
  • Lemma 1
  • proof
  • Lemma 2
  • ...and 12 more