A Resolution Independent Neural Operator

TL;DR

A general framework for operator learning from input-output data with arbitrary sensor locations and counts is introduced, and a resolution-independent DeepONet (RI-DeepONet), which handles input functions discretized arbitrarily but sufficiently finely, is introduced.

Abstract

The Deep Operator Network (DeepONet) is a powerful neural operator architecture that uses two neural networks to map between infinite-dimensional function spaces. This architecture allows for the evaluation of the solution field at any location within the domain but requires input functions to be discretized at identical locations, limiting practical applications. We introduce a general framework for operator learning from input-output data with arbitrary sensor locations and counts. This begins by introducing a resolution-independent DeepONet (RI-DeepONet), which handles input functions discretized arbitrarily but sufficiently finely. To achieve this, we propose two dictionary learning algorithms that adaptively learn continuous basis functions, parameterized as implicit neural representations (INRs), from correlated signals on arbitrary point clouds. These basis functions project input function data onto a finite-dimensional embedding space, making it compatible with DeepONet without architectural changes. We specifically use sinusoidal representation networks (SIRENs) as trainable INR basis functions. Similarly, the dictionary learning algorithms identify basis functions for output data, defining a new neural operator architecture: the Resolution Independent Neural Operator (RINO). In RINO, the operator learning task reduces to mapping coefficients of input basis functions to output basis functions. We demonstrate RINO's robustness and applicability in handling arbitrarily sampled input and output functions during both training and inference through several numerical examples.

Figures (39)

• Figure 1: Vanilla DeepONet and RI-DeepONet in a nutshell. Vanilla DeepONet requires input functions to be sampled at a fixed set of points (A.1), while RI-DeepONet allows for arbitrary discretization of the input functions (B.1). The branch network in Vanilla DeepONet operates directly on the discretized input functions (A.2), whereas in RI-DeepONet, the DeepONet operates on the embeddings obtained from projecting the input signal onto a dictionary learned a priori (B.2).
• Figure 2: Operator learning with RINO when the output basis functions used to span the output field are either (a) unknown or (b) fixed during the operator learning stage. In (a), only the input functions are embedded using a learned set of basis function and the operator is learned between the embedding space and the output directly by training both branch ($\mathcal{F}_{\boldsymbol{\theta}}^{\text{br}}$) and trunk ($\mathcal{F}_{\boldsymbol{\theta}}^{\text{br}}$) networks. In (b), is functions are learned separately for both output and input function data. Consequently, the operator learning task effectively becomes a mapping between these two constructed subspaces via a single nueral network $\mathcal{F}_{\boldsymbol{\theta}}^{\text{ri}}$.
• Figure 3: Dictionary learning (top) in the classical vector space setup and (bottom) in the proposed function space setup. The proposed method can be directly applied to point cloud signals that are sampled arbitrarily (but sufficiently richly) in terms of the number and location of discretized points. In the proposed method, classical discrete atoms (basis vectors) are replaced by continuous, fully differentiable functions parameterized by neural networks.
• Figure 4: Antiderivate Example: (a) Loss function during optimization iterations for dictionary learning of the input function bases. (b) Distribution of reconstruction errors for the train and test datasets after training. (c) The learned basis functions in the dictionary.
• Figure 5: Antiderivative Example: (a) Loss function during optimization iterations for operator learning. (b) Distribution of output function prediction errors for the training and testing datasets after training.

Theorems & Definitions (5)

• Remark 2.1
• Remark 2.2
• Definition 2.3: Orthogonal Functions
• Remark 2.4
• Remark 2.5