Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Basis-to-Basis Operator Learning Using Function Encoders

Tyler Ingebrand, Adam J. Thorpe, Somdatta Goswami, Krishna Kumar, Ufuk Topcu

TL;DR

Basis-to-Basis (B2B) operator learning is presented, a novel approach for learning operators on Hilbert spaces of functions based on the foundational ideas of function encoders that demonstrates a two-orders-of-magnitude improvement in accuracy over existing approaches on several benchmark tasks.

Abstract

We present Basis-to-Basis (B2B) operator learning, a novel approach for learning operators on Hilbert spaces of functions based on the foundational ideas of function encoders. We decompose the task of learning operators into two parts: learning sets of basis functions for both the input and output spaces and learning a potentially nonlinear mapping between the coefficients of the basis functions. B2B operator learning circumvents many challenges of prior works, such as requiring data to be at fixed locations, by leveraging classic techniques such as least squares to compute the coefficients. It is especially potent for linear operators, where we compute a mapping between bases as a single matrix transformation with a closed-form solution. Furthermore, with minimal modifications and using the deep theoretical connections between function encoders and functional analysis, we derive operator learning algorithms that are directly analogous to eigen-decomposition and singular value decomposition. We empirically validate B2B operator learning on seven benchmark operator learning tasks and show that it demonstrates a two-orders-of-magnitude improvement in accuracy over existing approaches on several benchmark tasks.

Basis-to-Basis Operator Learning Using Function Encoders

TL;DR

Basis-to-Basis (B2B) operator learning is presented, a novel approach for learning operators on Hilbert spaces of functions based on the foundational ideas of function encoders that demonstrates a two-orders-of-magnitude improvement in accuracy over existing approaches on several benchmark tasks.

Abstract

We present Basis-to-Basis (B2B) operator learning, a novel approach for learning operators on Hilbert spaces of functions based on the foundational ideas of function encoders. We decompose the task of learning operators into two parts: learning sets of basis functions for both the input and output spaces and learning a potentially nonlinear mapping between the coefficients of the basis functions. B2B operator learning circumvents many challenges of prior works, such as requiring data to be at fixed locations, by leveraging classic techniques such as least squares to compute the coefficients. It is especially potent for linear operators, where we compute a mapping between bases as a single matrix transformation with a closed-form solution. Furthermore, with minimal modifications and using the deep theoretical connections between function encoders and functional analysis, we derive operator learning algorithms that are directly analogous to eigen-decomposition and singular value decomposition. We empirically validate B2B operator learning on seven benchmark operator learning tasks and show that it demonstrates a two-orders-of-magnitude improvement in accuracy over existing approaches on several benchmark tasks.
Paper Structure (24 sections, 1 theorem, 25 equations, 24 figures, 4 tables, 3 algorithms)

This paper contains 24 sections, 1 theorem, 25 equations, 24 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Learning Nonlinear Operators Using Function Encoders
  4. Learning a Basis for Input and Output Domains
  5. Nonlinear Operators Between Learned Spaces
  6. Learning Linear Operators Via Least Squares
  7. Singular Value Decomposition and Eigendecompositions
  8. Using Function Encoders for Singular Value Decomposition (SVD)
  9. Using Function Encoders for Eigen-decompositions (ED)
  10. Results and Discussion
  11. Derivative and Anti-Derivative Operators
  12. 1D Darcy Flow
  13. 2D Darcy Flow
  14. Elastic Plate
  15. Parameterized Heat Equation
  16. ...and 9 more sections

Key Result

Theorem 1

If $f_3 := a f_1 + b f_2$, $a,b\in \mathbb{R}$, $f_1, f_2 \in \mathcal{G}$, and $\mathcal{T}$ is a linear operator, then $\hat{\mathcal{T}}f_3 = a \hat{\mathcal{T}}f_1 + b \hat{\mathcal{T}}f_2$.

Figures (24)

  • Figure 1: The basis-to-basis (B2B) operator maps the coefficients $\alpha$ of the input function $f$ to the coefficients $\beta$ of the output function $\mathcal{T}f$.
  • Figure 2: The function encoder training procedure. We compute estimates $\hat{f}_{1}, \ldots, \hat{f}_{n}$ of multiple functions $f_{1}, \ldots, f_{n} \in \mathcal{F}$ using data. Then, we compute the total loss of the overall functions and backpropagate the gradient with respect to the basis function parameters $\theta$ using gradient descent.
  • Figure 3: The function encoder inference procedure. Using online data from a new, unseen function $f_{new}$, we compute the corresponding coefficients $\alpha \in \mathbb{R}^{k}$ using least squares. The estimate $\hat{f}_{new}$ of $f_{new}$ is a linear combination of the fixed basis functions $\lbrace g_{j} \rbrace$ with the coefficients $\alpha$.
  • Figure 4: Left: The inference procedure for SVD. A dataset $D=\{(x_i, f(x_i))\}_{i=1}^m$ is given which describes a function $f$. This data is used to compute the coefficients of the basis functions via the least-squares solution. Then, $\hat{\mathcal{T}}f$ is approximated using \ref{['eqn: svd operator']}. Right: The inference procedure for ED. The procedure is analogous, with the only change being that $\mathcal{T}$ is self-adjoint, and so the same basis is used for the input and output spaces.
  • Figure 5: Decay curves for the derivative operator. (a) Shows the decay of eigenvalues from the ED approach. (b) Illustrates the decay of the singular values computed via the traditional singular value decomposition of the B2B matrix representation of the operator. Both plots demonstrate the rapid decay characteristic of compact operators.
  • ...and 19 more figures

Theorems & Definitions (2)

  • Theorem 1
  • proof