$C^*$-Algebraic Machine Learning: Moving in a New Direction

TL;DR

The paper proposes a novel framework, C*-algebraic machine learning, to unify and extend kernel methods and neural networks for structured data. By replacing scalar-valued constructs with C*-algebra-valued counterparts, it introduces Reproducing Kernel Hilbert C*-modules (RKHM), A-valued inner products on Hilbert C*-modules, and C*-algebra nets that enable joint, interactive representations of multiple models. It shows how kernel mean embeddings, deep learning with kernels, and multitask/ensemble settings can be reformulated within this algebraic language, and discusses expressiveness and optimization advantages of such nets. The work also outlines practical challenges, including computational cost, lack of inverses, and the handling of infinite-dimensional spaces, and points to future avenues such as Cuntz and AF algebras, quantum ML, and group-equivariant architectures. Overall, it broadens the mathematical toolkit available for ML, aiming to better handle complex data structures and model interactions while opening doors to quantum-inspired and noncommutative learning paradigms.

Abstract

Machine learning has a long collaborative tradition with several fields of mathematics, such as statistics, probability and linear algebra. We propose a new direction for machine learning research: $C^*$-algebraic ML $-$ a cross-fertilization between $C^*$-algebra and machine learning. The mathematical concept of $C^*$-algebra is a natural generalization of the space of complex numbers. It enables us to unify existing learning strategies, and construct a new framework for more diverse and information-rich data models. We explain why and how to use $C^*$-algebras in machine learning, and provide technical considerations that go into the design of $C^*$-algebraic learning models in the contexts of kernel methods and neural networks. Furthermore, we discuss open questions and challenges in $C^*$-algebraic ML and give our thoughts for future development and applications.

Figures (4)

• Figure 1: Overview of the $C^*$-algebraic machine learning
• Figure 2: Overview of kernel methods with RKHMs by hashimoto23-aistats. Here, $\mathcal{A}_1$ and $\mathcal{A}_2$ are $C^*$-algebras and $a_{i,j}$ is the parameter of the $C^*$-algebra-valued positive definite kernel associated with the feature maps $\phi_1$ and $\phi_2$. If $\mathcal{A}_1\subseteq \mathcal{A}_2$, then the RKHM over $\mathcal{A}_1$ is contained in the RKHM over $\mathcal{A}_2$.
• Figure 3: Overview of $C^*$-algebra net by hashimoto22. They focused on the $C^*$-algebra $C(\mathcal{Z})$ for a compact Hausdorff space $\mathcal{Z}$ and generalized neural network parameters to $C^*$-algebra-valued. We can continuously combine multiple (real-valued) neural networks using a single $C^*$-algebra net.
• Figure 4: Overview of the averaged $C^*$-algebra net $A_Pf(x)$. Here, $f_z(x)=\hat{\sigma}_1(\sum_{i=1}^3\alpha^1_{i,1}(z)x_i)$. We can regard $A_Pf(x)$ as a continuation of the $2$-layer neural network $\sum_{i=1}^{d_{2}}p_if_{z_i}(x)$.

Theorems & Definitions (15)

• Definition 2.1: $C^*$-algebra
• Definition 2.2: Positive
• Example 2.3
• Definition 4.1: $C^*$-module
• Definition 4.2: $\mathcal{A}$-valued inner product
• Definition 4.3: $\mathcal{A}$-valued absolute value and norm
• Definition 4.4: Hilbert $C^*$-module
• Definition 5.1: $\mathcal{A}$-valued positive definite kernel
• Proposition 5.2
• Proposition A.1