Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On finite-dimensional encoding/decoding theorems for neural operators

Vinícius Luz Oliveira, Vladimir G. Pestov

TL;DR

The work generalizes finite-dimensional encoding/decoding theorems to neural operator-style mappings between arbitrary locally convex spaces, showing that any continuous $f:E\to F$ can be approximated on compacta by maps factoring through two finite-dimensional spaces via $f\approx S\circ g\circ T$. It extends this to $C^k$-smooth mappings, establishing an equivalence with the approximation property of $E$ and thereby highlighting the central role of AP in the smooth setting. The results remove normability restrictions, making the theory applicable to non-normable spaces common in differential equations, which underpin neural-operator-inspired models. Collectively, the findings provide a robust mathematical foundation for finite-dimensional encoding/decoding schemes in infinite-dimensional neural operator frameworks, enabling practical computations without restrictive assumptions on the input/output spaces.

Abstract

Recently, versions of neural networks with infinite-dimensional affine operators inside the computational units (``neural operator'' networks) have been applied to learn solutions to differential equations. To enable practical computations, one employs finite-dimensional encoding/decoding theorems of the following kind: every continuous mapping $f$ between function spaces $E$ and $F$ is approximated in the topology of uniform convergence on compacta by continuous mappings factoring through two finite dimensional Banach spaces. Such a result is known (Kovachki et al., 2023) for $E,F$ being Banach spaces having the approximation property. We point out that the result needs no assumptions on $E,F$ whatsoever and remains true not only for all normed spaces, but for arbitrary locally convex spaces as well. At the same time, an analogous result for $C^k$-smooth mappings and the $C^k$ compact open topology, $k\geq 1$, holds if and only if the space $E$ has the approximation property. This analysis may be useful already because non-normable locally convex function spaces are common in the theory of differential equations, the main field of applications for the emerging theory.

On finite-dimensional encoding/decoding theorems for neural operators

TL;DR

The work generalizes finite-dimensional encoding/decoding theorems to neural operator-style mappings between arbitrary locally convex spaces, showing that any continuous can be approximated on compacta by maps factoring through two finite-dimensional spaces via . It extends this to -smooth mappings, establishing an equivalence with the approximation property of and thereby highlighting the central role of AP in the smooth setting. The results remove normability restrictions, making the theory applicable to non-normable spaces common in differential equations, which underpin neural-operator-inspired models. Collectively, the findings provide a robust mathematical foundation for finite-dimensional encoding/decoding schemes in infinite-dimensional neural operator frameworks, enabling practical computations without restrictive assumptions on the input/output spaces.

Abstract

Recently, versions of neural networks with infinite-dimensional affine operators inside the computational units (``neural operator'' networks) have been applied to learn solutions to differential equations. To enable practical computations, one employs finite-dimensional encoding/decoding theorems of the following kind: every continuous mapping between function spaces and is approximated in the topology of uniform convergence on compacta by continuous mappings factoring through two finite dimensional Banach spaces. Such a result is known (Kovachki et al., 2023) for being Banach spaces having the approximation property. We point out that the result needs no assumptions on whatsoever and remains true not only for all normed spaces, but for arbitrary locally convex spaces as well. At the same time, an analogous result for -smooth mappings and the compact open topology, , holds if and only if the space has the approximation property. This analysis may be useful already because non-normable locally convex function spaces are common in the theory of differential equations, the main field of applications for the emerging theory.
Paper Structure (14 sections, 24 theorems, 51 equations, 1 figure)

This paper contains 14 sections, 24 theorems, 51 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Background from functional analysis and topology
  3. Some notions and results of general topology
  4. Normed and Hilbert spaces
  5. Locally convex spaces
  6. Uniform continuity
  7. Calculus in locally convex spaces
  8. Tensor products
  9. Encoding/decoding of continuous functions
  10. Case of normed spaces
  11. Case of general locally convex spaces
  12. Smooth decoding/encoding
  13. (1)$\Rightarrow$(2): Necessity of the Approximation Property
  14. (2)$\Rightarrow$(1): Sufficiency of the Approximation Property

Key Result

Theorem 1.1

Let $f\colon E\to F$ be a continuous mapping between two locally convex spaces. It is approximated in the topology of uniform convergence on compacta by mappings of the form $S\circ g\circ T$, where $S\colon F_1\to F$ and $T\colon R\to E_1$ are bounded linear operators, $E_1,F_1$ are finite-dimensio

Figures (1)

  • Figure 1: Decoding/encoding

Theorems & Definitions (31)

  • Theorem 1.1: Finite encoding/decoding for neural operators, continuous case
  • Theorem 1.2: Finite encoding/decoding for neural operators, smooth case
  • Proposition 2.1: Th. 4.1.18 in engelking
  • Proposition 2.2: Th. 3.1.12 in engelking
  • Proposition 2.3: Corol. 2.4.8, engelking
  • Proposition 2.4: Th. 3.1.13, engelking
  • Proposition 2.5: Prop. 2.4.2, engelking
  • proof
  • Theorem 2.6
  • Theorem 2.7: Banach--Mazur theorem, banach, p. 185
  • ...and 21 more