Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spectral Truncation Kernels: Noncommutativity in $C^*$-algebraic Kernel Machines

Yuka Hashimoto, Ayoub Hafid, Masahiro Ikeda, Hachem Kadri

Abstract

$C^*$-algebra-valued kernels could pave the way for the next generation of kernel machines. To further our fundamental understanding of learning with $C^*$-algebraic kernels, we propose a new class of positive definite kernels based on the spectral truncation. We focus on kernels whose inputs and outputs are vectors or functions and generalize typical kernels by introducing the noncommutativity of the products appearing in the kernels. The noncommutativity induces interactions along the data function domain. We show that the proposed kernels fill the gap between existing separable and commutative kernels. We also propose a deep learning perspective to obtain a more flexible framework. The flexibility of the proposed class of kernels allows us to go beyond previous separable and commutative kernels, addressing two of the foremost issues regarding learning in vector-valued RKHSs, namely the choice of the kernel and the computational cost.

Spectral Truncation Kernels: Noncommutativity in $C^*$-algebraic Kernel Machines

Abstract

-algebra-valued kernels could pave the way for the next generation of kernel machines. To further our fundamental understanding of learning with -algebraic kernels, we propose a new class of positive definite kernels based on the spectral truncation. We focus on kernels whose inputs and outputs are vectors or functions and generalize typical kernels by introducing the noncommutativity of the products appearing in the kernels. The noncommutativity induces interactions along the data function domain. We show that the proposed kernels fill the gap between existing separable and commutative kernels. We also propose a deep learning perspective to obtain a more flexible framework. The flexibility of the proposed class of kernels allows us to go beyond previous separable and commutative kernels, addressing two of the foremost issues regarding learning in vector-valued RKHSs, namely the choice of the kernel and the computational cost.
Paper Structure (37 sections, 17 theorems, 45 equations, 8 figures, 2 tables)

This paper contains 37 sections, 17 theorems, 45 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. $C^*$-algebra and Reproducing kernel Hilbert $C^*$-module
  4. Spectral truncation on the torus
  5. Existing operator-valued kernels
  6. Separable kernel
  7. Commutative kernel
  8. Transformable kernel
  9. Combination of separable and transformable kernels
  10. $C^*$-algebra-valued Positive Definite Kernel with Spectral Truncation
  11. Connection between the proposed kernel and existing kernels
  12. The case of $n=1$
  13. The case of $n=\infty$
  14. The case of $1<n<\infty$
  15. Computational cost
  16. ...and 22 more sections

Key Result

Proposition 5

For each $z\in\mathbb{T}$, $x\ast F_n(z)\to x(z)$ as $n\to\infty$.

Figures (8)

  • Figure 1: Overview of the construction of the simplest kernel $k_n^{\operatorname{poly},1}(x,y)=S_n(R_n(x)^*R_n(y))$
  • Figure 2: Féjer kernel $F_n^{2,P}$ for $n=5,10,15$
  • Figure 3: (a,b) Test error of the regression task with different values of $n$. (Box plot of results of five independent runs with different values of noises $\xi_{z,i}$ and $\eta_{z,i}$ in $x^i(z)$ and $y^i(z)$.) (c) Eigenvalues of the Gram matrix $\mathbf{G}(0)$ indexed as the descending order for the regression task. (Average value of results of five different runs. The error bar represents the standard deviation.)
  • Figure 4: Test error of the regression task with deep approach with different $n$ and $L$. The parameters $n$ and $L$ are chosen so that the numbers of parameters are the same for all the cases. (Average value $\pm$ the standard deviation of three independent runs.)
  • Figure 5: (a) Test error of the image recovering task with different values of $n$ and the kernel $k=\hat{k}_n^{\operatorname{prod},q}$. (Box plot of results of five independent runs with different training and test data.) (b) Eigenvalues of the Gram matrix $\mathbf{G}(0)$ indexed as the descending order for the image recovering task. (Average value of results of five different runs. The error bar represents the standard deviation.)
  • ...and 3 more figures

Theorems & Definitions (31)

  • Definition 1: $C^*$-algebra
  • Example 1
  • Definition 2: Positive
  • Definition 3: Infimum and minimum
  • Definition 4: $\mathcal{A}$-valued positive definite kernel
  • Proposition 5
  • Example 2
  • Definition 6
  • Remark 7
  • Remark 8
  • ...and 21 more