Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Theory to Application: A Practical Introduction to Neural Operators in Scientific Computing

Prashant K. Jha

TL;DR

This work provides a practical, implementation-focused survey of neural operator architectures—DeepONet, PCANet, and Fourier Neural Operator—for learning forward operators $F: M \to U$ of parametric PDEs. Using two linear model problems (Poisson with diffusivity and in-plane elasticity with variable Young's modulus), it demonstrates finite element-based data generation, Gaussian-function-space priors, and SVD-driven dimensionality reduction to enable efficient learning. The paper then shows how neural operators can serve as fast surrogates within Bayesian inversion (via MCMC) to infer parameter fields, achieving posterior estimates close to those obtained with the true forward model on these problems. It further discusses challenges in accuracy and generalization, and outlines strategies such as residual-based error correction and multi-level training, while outlining future directions including PINO, uncertainty quantification, and multi-fidelity integration for robust, real-world deployment.

Abstract

This focused review explores a range of neural operator architectures for approximating solutions to parametric partial differential equations (PDEs), emphasizing high-level concepts and practical implementation strategies. The study covers foundational models such as Deep Operator Networks (DeepONet), Principal Component Analysis-based Neural Networks (PCANet), and Fourier Neural Operators (FNO), providing comparative insights into their core methodologies and performance. These architectures are demonstrated on two classical linear parametric PDEs: the Poisson equation and linear elastic deformation. Beyond forward problem-solving, the review delves into applying neural operators as surrogates in Bayesian inference problems, showcasing their effectiveness in accelerating posterior inference while maintaining accuracy. The paper concludes by discussing current challenges, particularly in controlling prediction accuracy and generalization. It outlines emerging strategies to address these issues, such as residual-based error correction and multi-level training. This review can be seen as a comprehensive guide to implementing neural operators and integrating them into scientific computing workflows.

From Theory to Application: A Practical Introduction to Neural Operators in Scientific Computing

TL;DR

This work provides a practical, implementation-focused survey of neural operator architectures—DeepONet, PCANet, and Fourier Neural Operator—for learning forward operators of parametric PDEs. Using two linear model problems (Poisson with diffusivity and in-plane elasticity with variable Young's modulus), it demonstrates finite element-based data generation, Gaussian-function-space priors, and SVD-driven dimensionality reduction to enable efficient learning. The paper then shows how neural operators can serve as fast surrogates within Bayesian inversion (via MCMC) to infer parameter fields, achieving posterior estimates close to those obtained with the true forward model on these problems. It further discusses challenges in accuracy and generalization, and outlines strategies such as residual-based error correction and multi-level training, while outlining future directions including PINO, uncertainty quantification, and multi-fidelity integration for robust, real-world deployment.

Abstract

This focused review explores a range of neural operator architectures for approximating solutions to parametric partial differential equations (PDEs), emphasizing high-level concepts and practical implementation strategies. The study covers foundational models such as Deep Operator Networks (DeepONet), Principal Component Analysis-based Neural Networks (PCANet), and Fourier Neural Operators (FNO), providing comparative insights into their core methodologies and performance. These architectures are demonstrated on two classical linear parametric PDEs: the Poisson equation and linear elastic deformation. Beyond forward problem-solving, the review delves into applying neural operators as surrogates in Bayesian inference problems, showcasing their effectiveness in accelerating posterior inference while maintaining accuracy. The paper concludes by discussing current challenges, particularly in controlling prediction accuracy and generalization. It outlines emerging strategies to address these issues, such as residual-based error correction and multi-level training. This review can be seen as a comprehensive guide to implementing neural operators and integrating them into scientific computing workflows.

Paper Structure

This paper contains 38 sections, 49 equations, 17 figures, 1 table.

Table of Contents

  1. Introduction
  2. Organization of the article
  3. Preliminaries
  4. Notations
  5. Series representation of functions and finite-dimensional approximation
  6. Finite element approximation
  7. Dimensional reduction and singular-value decomposition (SVD)
  8. Projectors via SVD
  9. Probability sampling of functions aka infinite-dimensional random variables
  10. Gaussian measures based on Laplacian-like operators
  11. Model problems
  12. Poisson problem
  13. Setup details and data generation
  14. Linear elasticity problem
  15. Setup details and data generation
  16. ...and 23 more sections

Figures (17)

  • Figure 1: Singular values of input and output data (centered and normalized) $\{{\bm{\mathrm{m}}}_i = \exp({\bm{\mathrm{w}}}_i) | {\bm{\mathrm{w}}}_i \sim {\bm{\mathrm{N}}}({\bm{\mathrm{0}}}, {\bm{\mathrm{C}}})\}$, and $\{{\bm{\mathrm{u}}}_i = {\bm{\mathrm{F}}}({\bm{\mathrm{m}}}_i)\}$, where ${\bm{\mathrm{N}}}({\bm{\mathrm{0}}}, {\bm{\mathrm{C}}})$ is the $p_m$-dimensional Gaussian density obtained via the finite element approximation of the random Gaussian field in function space $M:= L^2(D_m; \mathbb{R})$ (see \ref{['ss:sampling']} for details), ${\bm{\mathrm{m}}}_i$ discretized input to the parametric PDE, and ${\bm{\mathrm{F}}}$ a discretized solution operator associated with the PDE. $\sigma^a$, $a\in \{m, u\}$, represents the normalized singular values. Small dots show corresponding modes when the normalized singular value is $0.01$ or $0.1$. The dimension of the reduced space is $100$, and the grey dots show the corresponding singular value in the plot.
  • Figure 2: Random samples $w$ using Gaussian measure based on a Laplacian-like operator $L_\Delta$ defined in \ref{['eq:LDelta']}.
  • Figure 3: Some representative data samples for Poisson (a) and linear elasticity (b) problems.
  • Figure 4: Schematics of three neural operators, DeepONet, PCANet, and FNO. Grey and light green circles represent the input and output of neural operators. The blue box includes a parameterized neural network-based map. In this work, the blue boxes for DeepONet and PCANet employ multilayer perceptron fully-connected neural networks. In the case of FNO, trainable parameters (namely, $R_l, W_l, b_l$) appear within each Fourier layer.
  • Figure 5: Comparing DeepONet, PCANet, and FNO predictions with the finite element solution for four random samples of diffusivity in the Poisson problem. Here, $e$ is the relative percentage $l^2$ error corresponding to the sample.
  • ...and 12 more figures

Theorems & Definitions (1)

  • Remark 1