Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Operator-Grounded Continuous Tensor Function Representation and Its Applications

Ruoyang Su, Xi-Le Zhao, Sheng Liu, Wei-Hao Wu, Yisi Luo, Michael K. Ng

TL;DR

A neural operator-grounded continuous tensor function representation (abbreviated as NO-CTR) is proposed, which can more faithfully represent complex real-world data compared with classic discrete tensor representations and continuous tensor function representations.

Abstract

Recently, continuous tensor functions have attracted increasing attention, because they can unifiedly represent data both on mesh grids and beyond mesh grids. However, since mode-$n$ product is essentially discrete and linear, the potential of current continuous tensor function representations is still locked. To break this bottleneck, we suggest neural operator-grounded mode-$n$ operators as a continuous and nonlinear alternative of discrete and linear mode-$n$ product. Instead of mapping the discrete core tensor to the discrete target tensor, proposed mode-$n$ operator directly maps the continuous core tensor function to the continuous target tensor function, which provides a genuine continuous representation of real-world data and can ameliorate discretization artifacts. Empowering with continuous and nonlinear mode-$n$ operators, we propose a neural operator-grounded continuous tensor function representation (abbreviated as NO-CTR), which can more faithfully represent complex real-world data compared with classic discrete tensor representations and continuous tensor function representations. Theoretically, we also prove that any continuous tensor function can be approximated by NO-CTR. To examine the capability of NO-CTR, we suggest an NO-CTR-based multi-dimensional data completion model. Extensive experiments across various data on regular mesh grids (multi-spectral images and color videos), on mesh girds with different resolutions (Sentinel-2 images) and beyond mesh grids (point clouds) demonstrate the superiority of NO-CTR.

Neural Operator-Grounded Continuous Tensor Function Representation and Its Applications

TL;DR

A neural operator-grounded continuous tensor function representation (abbreviated as NO-CTR) is proposed, which can more faithfully represent complex real-world data compared with classic discrete tensor representations and continuous tensor function representations.

Abstract

Recently, continuous tensor functions have attracted increasing attention, because they can unifiedly represent data both on mesh grids and beyond mesh grids. However, since mode- product is essentially discrete and linear, the potential of current continuous tensor function representations is still locked. To break this bottleneck, we suggest neural operator-grounded mode- operators as a continuous and nonlinear alternative of discrete and linear mode- product. Instead of mapping the discrete core tensor to the discrete target tensor, proposed mode- operator directly maps the continuous core tensor function to the continuous target tensor function, which provides a genuine continuous representation of real-world data and can ameliorate discretization artifacts. Empowering with continuous and nonlinear mode- operators, we propose a neural operator-grounded continuous tensor function representation (abbreviated as NO-CTR), which can more faithfully represent complex real-world data compared with classic discrete tensor representations and continuous tensor function representations. Theoretically, we also prove that any continuous tensor function can be approximated by NO-CTR. To examine the capability of NO-CTR, we suggest an NO-CTR-based multi-dimensional data completion model. Extensive experiments across various data on regular mesh grids (multi-spectral images and color videos), on mesh girds with different resolutions (Sentinel-2 images) and beyond mesh grids (point clouds) demonstrate the superiority of NO-CTR.
Paper Structure (28 sections, 3 theorems, 33 equations, 9 figures, 8 tables)

This paper contains 28 sections, 3 theorems, 33 equations, 9 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Continuous Tensor Function Representations
  4. Neural Operators
  5. Notations and Preliminaries
  6. Revisit of Mode-$n$ Product
  7. Neural Operator-Grounded Continuous Tensor Function Representation
  8. Continuous and Nonlinear Mode-$n$ Operators
  9. Continuous Tensor Function Representation
  10. Implementations
  11. Continuous Core Tensor Function
  12. Continuous and Nonlinear Mode-$n$ Operators
  13. Universal Approximation Theorem
  14. Multi-Dimensional Data Completion Model
  15. Experiments
  16. ...and 13 more sections

Key Result

Theorem 1

For any continuous tensor function $\boldsymbol{\mathsf{X}} \in C([0,1]^N)$, $\forall \varepsilon > 0$, there is a fully connected neural network and $N$ DeepONets lu2021learning such that where $\mathscr{F}_n$ is the continuous and nonlinear mode-$n$ operator induced by $F_n$.

Figures (9)

  • Figure 1: Discrete and linear mode-$n$ operator and proposed continuous and nonlinear mode-$n$ operator. The basic unit performed upon by discrete and linear mode-$n$ operator is the mode-$n$ fiber vectors of the discrete core tensor $\boldsymbol{\mathcal{G}}$, which are then mapped to the mode-$n$ fiber vectors of the discrete target tensor $\boldsymbol{\mathcal{X}}$. The basic unit performed upon by continuous and nonlinear mode-$n$ operator is the mode-$n$ univariate fiber functions of the continuous core tensor function $\boldsymbol{\mathsf{G}}$, which are then mapped to the mode-$n$ univariate fiber functions of the continuous target tensor function $\boldsymbol{\mathsf{X}}$.
  • Figure 2: Visual and quantitative recovery results with discrete and linear mode-$n$ operators (LRTFR luo2024low), continuous and nonlinear mode-$n$ operators (proposed NO-CTR) on the MSI Toy (top) and point cloud Frog (bottom) when the sampling rates are $10\%$.
  • Figure 3: Flow chart of NO-CTR and the corresponding multi-dimensional data completion model. We first cleverly leverage neural operators to suggest continuous and nonlinear mode-$n$ operators. Specifically, the basic unit performed upon by continuous and nonlinear mode-$n$ operator is the mode-$n$ univariate fiber functions of the continuous core tensor function $\boldsymbol{\mathsf{G}}$, which are then mapped to the mode-$n$ univariate fiber functions of the continuous target tensor function $\boldsymbol{\mathsf{X}}$. Empowering with continuous and nonlinear mode-$n$ operators, we propose a neural operator-grounded continuous tensor function representation (NO-CTR), which can faithfully represent continuous target tensor function $\boldsymbol{\mathsf{X}}$ as a continuous core tensor function $\boldsymbol{\mathsf{G}}$ and a series of continuous and nonlinear mode-$n$ operators $\{\mathscr{F}^{\langle n \rangle}\}_{n=1}^N$. To examine the capability of NO-CTR, we suggest an NO-CTR-based multi-dimensional data completion model.
  • Figure 4: Visual recovery results of MSI completion at a sampling rate of $10\%$.
  • Figure 5: Visual recovery results of color video completion at a sampling rate of $10\%$.
  • ...and 4 more figures

Theorems & Definitions (8)

  • Definition 1: Mode-$n$ Unfolding kolda2009tensor
  • Definition 2: Mode-$n$ Product kolda2009tensor
  • Definition 3: Discrete and Linear Mode-$n$ Operator
  • Definition 4: Continuous and Nonlinear Mode-$n$ Operator
  • Definition 5: NO-CTR
  • Theorem 1
  • Lemma 1: calin2020universal
  • Lemma 2: lu2021learningchen1995universal