Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fully tensorial approach to hypercomplex neural networks

Agnieszka Niemczynowicz, Radosław Antoni Kycia

TL;DR

The paper addresses incorporating hypercomplex algebras into neural networks by representing algebra multiplication as a rank-3 tensor $A_{ij}^{k}$ and integrating this tensor into dense and convolutional layers. It develops general architectures for hypercomplex dense and $k$-D convolutional layers that operate on arbitrary algebra dimensions, and proves a tensor-based universal approximation theorem for hypercomplex perceptrons in non-degenerate algebras. The approach extends prior 4D implementations (e.g., Marcos1, QuaternionicNN2), is compatible with modern tensor frameworks, and is implemented in Hypercomplex Keras, enabling efficient, scalable, and flexible algebraic neural computations. This tensor-centric formulation broadens the applicability of hypercomplex neural networks to data naturally encoded in high-dimensional algebraic elements and offers a practical path to leveraging tensor libraries for advanced algebraic neural models.

Abstract

Fully tensorial theory of hypercomplex neural networks is given. It allows neural networks to use arithmetic based on arbitrary algebras. The key point is to observe that algebra multiplication can be represented as a rank three tensor and use this tensor in every algebraic operation. This approach is attractive for neural network libraries that support effective tensorial operations. It agrees with previous implementations for four-dimensional algebras. The proof of Universal Approximation Theorem for tensor formalism was given.

Fully tensorial approach to hypercomplex neural networks

TL;DR

The paper addresses incorporating hypercomplex algebras into neural networks by representing algebra multiplication as a rank-3 tensor and integrating this tensor into dense and convolutional layers. It develops general architectures for hypercomplex dense and -D convolutional layers that operate on arbitrary algebra dimensions, and proves a tensor-based universal approximation theorem for hypercomplex perceptrons in non-degenerate algebras. The approach extends prior 4D implementations (e.g., Marcos1, QuaternionicNN2), is compatible with modern tensor frameworks, and is implemented in Hypercomplex Keras, enabling efficient, scalable, and flexible algebraic neural computations. This tensor-centric formulation broadens the applicability of hypercomplex neural networks to data naturally encoded in high-dimensional algebraic elements and offers a practical path to leveraging tensor libraries for advanced algebraic neural models.

Abstract

Fully tensorial theory of hypercomplex neural networks is given. It allows neural networks to use arithmetic based on arbitrary algebras. The key point is to observe that algebra multiplication can be represented as a rank three tensor and use this tensor in every algebraic operation. This approach is attractive for neural network libraries that support effective tensorial operations. It agrees with previous implementations for four-dimensional algebras. The proof of Universal Approximation Theorem for tensor formalism was given.
Paper Structure (9 sections, 3 theorems, 25 equations, 1 figure, 2 algorithms)

This paper contains 9 sections, 3 theorems, 25 equations, 1 figure, 2 algorithms.

Table of Contents

  1. Introduction
  2. Theory
  3. Tensors
  4. (Hypercomplex) Algebras
  5. Neural Networks Architectures
  6. Hypercomplex dense layer
  7. Hypercomplex convolutional layer
  8. Tensor version of universal approximation theorem for single layer of hypercomplex perceptrons
  9. Conclusion

Key Result

Lemma 1

For a Hilbert space $H$ over field $\mathbb{F}$ (e.g., $\mathbb{R}$ or $\mathbb{C}$) with inner product $<\_|\_>:X\times X \rightarrow \mathbb{F}$ every continous linear functional $\phi\in H^{*}$ there is a vector $v_{\phi}\in H$ such that for all $x\in H$.

Figures (1)

  • Figure 1: Unique factorization property of the tensor product means that $F = f\circ \otimes$.

Theorems & Definitions (8)

  • proof
  • proof
  • Definition 1
  • Lemma 1: Riesz's representation lemma
  • Lemma 2: Riesz's representation lemma for nondegenerate algebra
  • proof
  • Theorem 1
  • proof