Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Model Reduction and Neural Networks for Parametric PDEs

Kaushik Bhattacharya, Bamdad Hosseini, Nikola B. Kovachki, Andrew M. Stuart

TL;DR

This work develops a mesh-independent, data-driven framework for learning solution operators of parametric PDEs by combining PCA-based dimension reduction in function spaces with neural networks operating on latent codes. It provides rigorous PCA-generalization bounds and neural-network approximation guarantees, showing that, under suitable measures, the overall map can be approximated to arbitrary accuracy while remaining robust to discretization. Numerical experiments on Darcy flow and Burgers' equation demonstrate strong performance both within and beyond the theoretical assumptions, including transfer across meshes and favorable online/offline trade-offs relative to intrusive methods. The approach presents a principled, non-intrusive alternative to traditional surrogate modeling for forward operators in PDE analysis, with potential extensions to nonlinear, time-dependent, and autoencoder-based function-space reductions.

Abstract

We develop a general framework for data-driven approximation of input-output maps between infinite-dimensional spaces. The proposed approach is motivated by the recent successes of neural networks and deep learning, in combination with ideas from model reduction. This combination results in a neural network approximation which, in principle, is defined on infinite-dimensional spaces and, in practice, is robust to the dimension of finite-dimensional approximations of these spaces required for computation. For a class of input-output maps, and suitably chosen probability measures on the inputs, we prove convergence of the proposed approximation methodology. We also include numerical experiments which demonstrate the effectiveness of the method, showing convergence and robustness of the approximation scheme with respect to the size of the discretization, and compare it with existing algorithms from the literature; our examples include the mapping from coefficient to solution in a divergence form elliptic partial differential equation (PDE) problem, and the solution operator for viscous Burgers' equation.

Model Reduction and Neural Networks for Parametric PDEs

TL;DR

This work develops a mesh-independent, data-driven framework for learning solution operators of parametric PDEs by combining PCA-based dimension reduction in function spaces with neural networks operating on latent codes. It provides rigorous PCA-generalization bounds and neural-network approximation guarantees, showing that, under suitable measures, the overall map can be approximated to arbitrary accuracy while remaining robust to discretization. Numerical experiments on Darcy flow and Burgers' equation demonstrate strong performance both within and beyond the theoretical assumptions, including transfer across meshes and favorable online/offline trade-offs relative to intrusive methods. The approach presents a principled, non-intrusive alternative to traditional surrogate modeling for forward operators in PDE analysis, with potential extensions to nonlinear, time-dependent, and autoencoder-based function-space reductions.

Abstract

We develop a general framework for data-driven approximation of input-output maps between infinite-dimensional spaces. The proposed approach is motivated by the recent successes of neural networks and deep learning, in combination with ideas from model reduction. This combination results in a neural network approximation which, in principle, is defined on infinite-dimensional spaces and, in practice, is robust to the dimension of finite-dimensional approximations of these spaces required for computation. For a class of input-output maps, and suitably chosen probability measures on the inputs, we prove convergence of the proposed approximation methodology. We also include numerical experiments which demonstrate the effectiveness of the method, showing convergence and robustness of the approximation scheme with respect to the size of the discretization, and compare it with existing algorithms from the literature; our examples include the mapping from coefficient to solution in a divergence form elliptic partial differential equation (PDE) problem, and the solution operator for viscous Burgers' equation.

Paper Structure

This paper contains 21 sections, 9 theorems, 104 equations, 13 figures.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Our Contribution
  4. Proposed Method
  5. Overview
  6. PCA On Function Space
  7. Neural Networks
  8. Comparison to Existing Methods
  9. Approximation Theory
  10. PCA And Approximation
  11. Neural Networks And Approximation
  12. Numerical Results
  13. PDE Setting
  14. Globally Lipschitz Solution Map
  15. Darcy Flow
  16. ...and 6 more sections

Key Result

theorem 1

Let $\mathcal{X}$, $\mathcal{Y}$ be real, separable Hilbert spaces and let $\mu$ be a probability measure supported on $\mathcal{X}$ such that $\mathbb{E}_{x \sim \mu} \|x\|_\mathcal{X}^4 < \infty$. Suppose $\Psi : \mathcal{X} \to \mathcal{Y}$ is a $\mu$-measurable, globally Lipschitz map. For any $

Figures (13)

  • Figure 1: A diagram summarizing various maps of interest in our proposed approach for the approximation of input-output maps between infinite-dimensional spaces.
  • Figure 2: Representative samples for each of the probability measures $\mu_{\text{G}}, \mu_{\text{L}}, \mu_{\text{P}}, \mu_{\text{B}}$ defined in Subsection \ref{['ssec:PDES']}. $\mu_{\text{G}}$ and $\mu_{\text{P}}$ are used in Subsection \ref{['sec:numlip']} to model the inputs, $\mu_{\text{L}}$ and $\mu_{\text{P}}$ are used in Subsection \ref{['sec:numdarcy']}, and $\mu_{\text{B}}$ is used in Subsection \ref{['sec:burgers']}.
  • Figure 3: Randomly chosen examples from the test set for each of the five considered problems. Each row is a different problem: linear elliptic, Poisson, Darcy flow with log-normal coefficients, Darcy flow with piecewise constant coefficients, and Burgers' equation respectively from top to bottom. The approximations are constructed with our best performing method (for $N=1024$): Linear $d=150$, Linear $d=150$, NN $d=70$, NN $d=70$, NN $d=15$ respectively from top to bottom.
  • Figure 4: Relative test errors on the linear elliptic problem. Using $N=1024$ training examples, panel (a) shows the errors as a function of the resolution while panel (b) fixes a $421 \times 421$ mesh and shows the error as a function of the reduced dimension. Panel (c) only shows results for our method using a neural network, fixing a $421 \times 421$ mesh and showing the error as a function of the reduced dimension for different amounts of training data.
  • Figure 5: Relative test errors on the Poisson problem. Using $N=1024$ training examples, panel (a) shows the errors as a function of the resolution while panel (b) fixes a $421 \times 421$ mesh and shows the error as a function of the reduced dimension. Panel (c) only shows results for our method using a neural network, fixing a $421 \times 421$ mesh and showing the error as a function of the reduced dimension for different amounts of training data.
  • ...and 8 more figures

Theorems & Definitions (19)

  • theorem 1
  • remark 1
  • remark 2
  • theorem 2
  • proof
  • theorem 3
  • proof
  • theorem 4
  • proof
  • lemma 1: Fan Fan
  • ...and 9 more