Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Universal Approximation Theorem for Neural Networks with Outputs in Locally Convex Spaces

Sachin Saini

TL;DR

It is shown that this class of neural networks is dense in the space of continuous mappings from a compact subset of the input space into the target space with respect to the topology of uniform convergence induced by the defining seminorms.

Abstract

In this paper, a universal approximation theorem (UAT) for shallow neural networks whose inputs belong to a topological vector space (TVS) and whose outputs take values in a Hausdorff locally convex TVS is established. The networks are constructed using scalar activation functions applied to continuous linear functionals of the input, while the output coefficients lie in the target space. It is shown that this class of neural networks is dense in the space of continuous mappings from a compact subset of the input space into the target space with respect to the topology of uniform convergence induced by the defining seminorms. The result extends existing scalar-valued approximation theorems on TVS and includes Banach and Hilbert-valued approximation as special cases. Several corollaries and examples are provided, illustrating applications to operator approximation between function spaces.

A Universal Approximation Theorem for Neural Networks with Outputs in Locally Convex Spaces

TL;DR

It is shown that this class of neural networks is dense in the space of continuous mappings from a compact subset of the input space into the target space with respect to the topology of uniform convergence induced by the defining seminorms.

Abstract

In this paper, a universal approximation theorem (UAT) for shallow neural networks whose inputs belong to a topological vector space (TVS) and whose outputs take values in a Hausdorff locally convex TVS is established. The networks are constructed using scalar activation functions applied to continuous linear functionals of the input, while the output coefficients lie in the target space. It is shown that this class of neural networks is dense in the space of continuous mappings from a compact subset of the input space into the target space with respect to the topology of uniform convergence induced by the defining seminorms. The result extends existing scalar-valued approximation theorems on TVS and includes Banach and Hilbert-valued approximation as special cases. Several corollaries and examples are provided, illustrating applications to operator approximation between function spaces.
Paper Structure (10 sections, 8 theorems, 42 equations, 1 figure)

This paper contains 10 sections, 8 theorems, 42 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Main Results
  3. Corollaries
  4. Relation to the Banach setting
  5. Applications to Operator Approximation
  6. Approximation of nonlinear operators
  7. Approximation of integral operators
  8. Connection with neural operator learning
  9. Approximation of solution operators for partial differential equations
  10. Conclusion

Key Result

Theorem 2.1

Assume that $S$ possesses the HBEP and let $E \subset S$ be compact. Let $\eta : \mathbb{R} \to \mathbb{R}$ be continuous and not a polynomial on any nonempty open interval. Define Then $\mathcal{A}_{S,T}^{\eta}$ is dense in $C(E;T)$ with respect to uniform convergence on $E$ induced by the continuous seminorms of $T$. Equivalently, for every $F \in C(E;T)$, every continuous seminorm $\rho$ on $T

Figures (1)

  • Figure 1: Architecture of the above-defined NNs.

Theorems & Definitions (17)

  • Theorem 2.1: Vector-valued UAT
  • Remark 2.2
  • Lemma 2.3
  • proof
  • Remark 2.4
  • Lemma 2.5
  • proof
  • proof : Proof of Theorem \ref{['thm:vector_uat']}
  • Remark 2.6: Dual formulation
  • Corollary 3.1: Hilbert-valued approximation
  • ...and 7 more