Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model Reduction for Operator Learning

Hao Liu, Biraj Dahal, Rongjie Lai, Wenjing Liao

TL;DR

This work addresses operator learning between infinite-dimensional function spaces by introducing AENet, a two-stage nonlinear model-reduction framework. It first learns a low-dimensional latent representation of inputs via an Auto-Encoder, then learns a latent-to-output transformation to approximate the solution operator. The authors establish an approximation theory guaranteeing neural networks can approximate the latent maps and forward operator, and prove a generalization bound where the sample complexity scales with the intrinsic dimension $d$, with robustness to noise. Empirically, AENet outperforms linear-reduction baselines and DeepONet on nonlinear PDE operators (transport, Burgers, KdV), and exhibits discretization-invariant performance under grid changes. The results provide a theoretical foundation for why nonlinear latent encoders can dramatically reduce sample complexity in operator learning and demonstrate practical benefits for physics-informed data-driven modeling.

Abstract

Many physical processes in science and engineering are naturally represented by operators between infinite-dimensional function spaces. The problem of operator learning, in this context, seeks to extract these physical processes from empirical data, which is challenging due to the infinite or high dimensionality of data. An integral component in addressing this challenge is model reduction, which reduces both the data dimensionality and problem size. In this paper, we utilize low-dimensional nonlinear structures in model reduction by investigating Auto-Encoder-based Neural Network (AENet). AENet first learns the latent variables of the input data and then learns the transformation from these latent variables to corresponding output data. Our numerical experiments validate the ability of AENet to accurately learn the solution operator of nonlinear partial differential equations. Furthermore, we establish a mathematical and statistical estimation theory that analyzes the generalization error of AENet. Our theoretical framework shows that the sample complexity of training AENet is intricately tied to the intrinsic dimension of the modeled process, while also demonstrating the remarkable resilience of AENet to noise.

Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model Reduction for Operator Learning

TL;DR

This work addresses operator learning between infinite-dimensional function spaces by introducing AENet, a two-stage nonlinear model-reduction framework. It first learns a low-dimensional latent representation of inputs via an Auto-Encoder, then learns a latent-to-output transformation to approximate the solution operator. The authors establish an approximation theory guaranteeing neural networks can approximate the latent maps and forward operator, and prove a generalization bound where the sample complexity scales with the intrinsic dimension , with robustness to noise. Empirically, AENet outperforms linear-reduction baselines and DeepONet on nonlinear PDE operators (transport, Burgers, KdV), and exhibits discretization-invariant performance under grid changes. The results provide a theoretical foundation for why nonlinear latent encoders can dramatically reduce sample complexity in operator learning and demonstrate practical benefits for physics-informed data-driven modeling.

Abstract

Many physical processes in science and engineering are naturally represented by operators between infinite-dimensional function spaces. The problem of operator learning, in this context, seeks to extract these physical processes from empirical data, which is challenging due to the infinite or high dimensionality of data. An integral component in addressing this challenge is model reduction, which reduces both the data dimensionality and problem size. In this paper, we utilize low-dimensional nonlinear structures in model reduction by investigating Auto-Encoder-based Neural Network (AENet). AENet first learns the latent variables of the input data and then learns the transformation from these latent variables to corresponding output data. Our numerical experiments validate the ability of AENet to accurately learn the solution operator of nonlinear partial differential equations. Furthermore, we establish a mathematical and statistical estimation theory that analyzes the generalization error of AENet. Our theoretical framework shows that the sample complexity of training AENet is intricately tied to the intrinsic dimension of the modeled process, while also demonstrating the remarkable resilience of AENet to noise.
Paper Structure (45 sections, 19 theorems, 164 equations, 12 figures, 1 table)

This paper contains 45 sections, 19 theorems, 164 equations, 12 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries and discretization of functions
  3. Notation
  4. Preliminaries
  5. Function spaces and discretization
  6. Nonlinear model reduction by AENet
  7. Problem formulation
  8. Low-dimensional nonlinear models
  9. AENet
  10. Main results on approximation and generalization errors
  11. Approximation theory of AENet
  12. Generalization error of AENet
  13. Numerical experiments
  14. Transport equation
  15. Burgers' equation
  16. ...and 30 more sections

Key Result

Lemma 1

In Setting setting1 and under Assumptions assumption:samplinglip and assumptiong:globalparametrization, every point in $\widetilde{\mathcal{M}}$ exhibits the low-dimensional parameterization: $\widetilde{\mathbf{f}}: \ \widetilde{\mathcal{M}} \subset \mathbb{R}^{D_1} \rightarrow [-1,1]^d,$ such that $\widetilde{\mathbf{f}}$ and $\widetilde{\mathbf{g}}$ are invertible and Lipschitz with Lipchitz co

Figures (12)

  • Figure 1: An illustration of the AENet architecture and the transformation flow chart. The oracle transformation $\Phi$ has a dimension reduction component $\widetilde{\mathbf{f}}$ and a forward transformation component $\Gamma$. These two components are marked in red.
  • Figure 2: Nonlinearity of the initial conditions for the transport equation. (a) shows the singular values of the data matrix. (b) shows the projection of data to the top 2 principal components and (c) shows the projection to top 3 principal components. (d) shows the projection to the 4th-6th principal components. In (b), (c) and (d), the projections are colored according to the $a$ parameter in the left subplots and according to the $h$ parameter in the right subplots.
  • Figure 3: Latent features of the initial conditions $g_{\alpha,h}$ (Transport) in \ref{['eq:ftransport']} given by the Auto-Encoder. The left plot is colored according to $a$ and the right plot is colored according to $h$.
  • Figure 4: Results of the transport equation.
  • Figure 5: $w_0$ and $w_1$ used for the Burgers' example
  • ...and 7 more figures

Theorems & Definitions (42)

  • Definition 1: Lipschitz operators
  • Definition 2: Minkowski dimension
  • Example 1
  • Lemma 1
  • Lemma 2
  • Theorem 1
  • Corollary 1
  • Remark 1
  • Theorem 2
  • Remark 2
  • ...and 32 more