Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nonlinear model reduction for operator learning

Hamidreza Eivazi, Stefan Wittek, Andreas Rausch

TL;DR

This work proposes an efficient framework that combines neural networks with kernel principal component analysis (KPCA) for operator learning and demonstrates the superior performance of KPCA-DeepONet over POD-DeepONet.

Abstract

Operator learning provides methods to approximate mappings between infinite-dimensional function spaces. Deep operator networks (DeepONets) are a notable architecture in this field. Recently, an extension of DeepONet based on model reduction and neural networks, proper orthogonal decomposition (POD)-DeepONet, has been able to outperform other architectures in terms of accuracy for several benchmark tests. We extend this idea towards nonlinear model order reduction by proposing an efficient framework that combines neural networks with kernel principal component analysis (KPCA) for operator learning. Our results demonstrate the superior performance of KPCA-DeepONet over POD-DeepONet.

Nonlinear model reduction for operator learning

TL;DR

This work proposes an efficient framework that combines neural networks with kernel principal component analysis (KPCA) for operator learning and demonstrates the superior performance of KPCA-DeepONet over POD-DeepONet.

Abstract

Operator learning provides methods to approximate mappings between infinite-dimensional function spaces. Deep operator networks (DeepONets) are a notable architecture in this field. Recently, an extension of DeepONet based on model reduction and neural networks, proper orthogonal decomposition (POD)-DeepONet, has been able to outperform other architectures in terms of accuracy for several benchmark tests. We extend this idea towards nonlinear model order reduction by proposing an efficient framework that combines neural networks with kernel principal component analysis (KPCA) for operator learning. Our results demonstrate the superior performance of KPCA-DeepONet over POD-DeepONet.
Paper Structure (18 sections, 1 theorem, 8 equations, 6 figures, 4 tables)

This paper contains 18 sections, 1 theorem, 8 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Operator learning.
  3. Our contributions.
  4. Methodology
  5. KPCA-DeepONet.
  6. Numerical experiments
  7. Conclusions
  8. Appendix
  9. Deep operator networks (DeepONets)
  10. KPCA-DeepONet diagram
  11. The representer theorem.
  12. Further experimental results
  13. Test setup
  14. Architecture and hyperparameters
  15. Kernels.
  16. ...and 3 more sections

Key Result

Theorem A.1

Representer theorem learningwithKernels: Let $\Omega: [0, +\infty) \mapsto \mathbb{R}$ be strictly increasing and let $L$ be a loss function. Consider the optimization problem where ${\mathcal{F}}$ is a reproducing kernel Hilbert spaces (RKHS) with kernel $k_z$, and $\lambda > 0$. Then, any optimal solution has the form of $h(\cdot) = \sum_{i = 1}^N \alpha_i k_z(\cdot, {\bm{z}}_i)$, where $\alpha

Figures (6)

  • Figure 1: Comparison of the proposed KPCA-DeepONet (orange, $\blacksquare$) and POD-DeepONet (blue, $\bullet$). Lines and shades indicate mean and standard deviation, respectively, over 5 independent trials.
  • Figure 2: A diagram of the KPCA-DeepONet operator learning setup summarizing various maps of interest. ${\mathcal{G}}$ is the operator we want to learn. ${\mathcal{P}}$ and ${\mathcal{Q}}$ are the evaluation operators, $b$ is the mapping by the branch network, $f$ is the projection on the KPCA basis, and $h$ is the mapping by the kernel ridge regression. The red color indicates those mappings that are only required for training.
  • Figure 3: KPCA-DeepONet prediction against the reference data for one sample of the test dataset for the Navier--Stokes equation. $\tilde{\cdot}$ indicates the KPCA-DeepONet prediction.
  • Figure 4: KPCA-DeepONet prediction against the reference data for one sample of the test dataset for the cavity flow. $\tilde{\cdot}$ indicates the KPCA-DeepONet prediction.
  • Figure 5: Performance of KPCA-DeepONet when utilizing a linear (orange, $\blacksquare$) and a quadratic (green, $\blacklozenge$) kernel for mapping to the latent space for the 1D nonlinear problem. Lines and shades indicate mean and standard deviation, respectively, over 5 independent trials.
  • ...and 1 more figures

Theorems & Definitions (1)

  • Theorem A.1