Universal Approximation Theorem for Vector- and Hypercomplex-Valued Neural Networks

TL;DR

The paper extends the universal approximation theorem (UAT) to vector-valued neural networks (V-nets) defined on finite-dimensional algebras, including hypercomplex algebras, by introducing non-degenerate algebras as a key condition. It proves that single-hidden-layer vector-valued MLPs with split Tauber-Wiener activations are dense in the space of continuous functions on compact sets, $\mathcal{H}_{\mathbb{V}}$ dense in $\mathcal{C}(K)$, when the underlying algebra is non-degenerate; hypercomplex outputs are covered as a special case. The authors generalize prior UAT results for complex, quaternion, tessarine, and Clifford networks within a unified ABIPNN-like bilinear-form framework and clarify how degeneracy (basis dependence) can obstruct universality. Numerical experiments on 2D and 4D algebras illustrate when universality holds (non-degenerate cases) and when it can fail (degenerate cases), supporting practical use of V-nets for multidimensional signals. Overall, the work provides a broad, theoretically grounded foundation for vector- and hypercomplex-valued networks in regression and classification tasks where multidimensional representations naturally arise.

Abstract

The universal approximation theorem states that a neural network with one hidden layer can approximate continuous functions on compact sets with any desired precision. This theorem supports using neural networks for various applications, including regression and classification tasks. Furthermore, it is valid for real-valued neural networks and some hypercomplex-valued neural networks such as complex-, quaternion-, tessarine-, and Clifford-valued neural networks. However, hypercomplex-valued neural networks are a type of vector-valued neural network defined on an algebra with additional algebraic or geometric properties. This paper extends the universal approximation theorem for a wide range of vector-valued neural networks, including hypercomplex-valued models as particular instances. Precisely, we introduce the concept of non-degenerate algebra and state the universal approximation theorem for neural networks defined on such algebras.

Figures (7)

• Figure 1: MSE by the number of epochs during the training phase of V-MLP networks with real-valued output weights defined on two-dimensional algebras.
• Figure 2: Surfaces of the components of the function $f_{\mathbb{V}}$ given by \ref{['eq:f-example-2D']} and the V-MLP networks with real-valued output weights defined on a two-dimensional degenerate algebra.
• Figure 3: Surfaces of the V-MLP networks with real-valued output weights defined on the two-dimensional algebras $\mathbb{D}$ (dual numbers) and $\mathbb{E}$ (equivalent to dual numbers).
• Figure 4: MSE by the number of epochs during the training phase of V-MLP networks with vector-valued output weights defined on non-degenerate two-dimensional algebras.
• Figure 5: Surfaces of the components of the V-MLP network, with vector-valued output weights, defined on the two-dimensional non-degenerate algebra $\mathbb{A}$.

Theorems & Definitions (17)

• Definition 1: Algebra Schafer1961AnAlgebras
• Definition 2: Non-degenerate Algebra
• Definition 3: Hypercomplex Algebra
• Lemma 1: Representation of Linear Functionals on Non-degenerate Algebras
• Remark 1
• proof
• Example 1
• Definition 4: Tauber-Wiener Functions
• Definition 5: Split Activation Function
• Remark 2