KHNNs: hypercomplex neural networks computations via Keras using TensorFlow and PyTorch

TL;DR

KHNN addresses the absence of a general framework for hypercomplex neural networks by introducing a Keras-based library with TensorFlow and PyTorch backends that implements Dense and Convolutional layers operating over arbitrary algebras. The approach relies on an Algebra module using structure constants $A_{ijk}$ with a basis $\{e_i\}$ so that $e_i e_j = A_{ijk} e_k$, enabling predefined algebras like Complex, Quaternions, Klein4, Clifford, Bicomplex, Tessarines, and Octonions, as well as easy extension to new ones. The authors present a three-part software architecture (Algebra module, TensorFlow-based layers, PyTorch-based layers) and illustrate practical usage with quaternion and 2D hypercomplex CNN examples, including a malaria image classification scenario. KHNN significantly lowers the barrier to applying hypercomplex representations by providing a cohesive, multi-backend framework that can accelerate research and application across domains. This framework opens pathways for broad adoption of hypercomplex NN and supports data encapsulation in algebraic structures, potentially reducing parameter counts while maintaining performance.

Abstract

Neural networks used in computations with more advanced algebras than real numbers perform better in some applications. However, there is no general framework for constructing hypercomplex neural networks. We propose a library integrated with Keras that can do computations within TensorFlow and PyTorch. It provides Dense and Convolutional 1D, 2D, and 3D layers architectures.

Figures (2)

• Figure 1: Accuracy for training and validation data during fitting the model.
• Figure 2: Loss function for training and validation data during fitting the model.