Geometric Learning with Positively Decomposable Kernels

TL;DR

This work tackles learning on non-Euclidean spaces where positive-definite kernels are hard to construct by adopting reproducing kernel Krein spaces ($RKKS$) that require only kernels with a $PD$ decomposition. It proves an $RKKS$ representer theorem that does not require explicit knowledge of the $PD$ decomposition when the regularizer is linear in the indefinite inner product, and develops a harmonic-analytic framework that links invariant kernels on $G/H$ to $PD$-decomposable functions on the double coset space $H\backslash G/H$. The paper provides verifiable sufficient conditions for $PD$ decomposability, including commutative (Abelian) cases where the Gaussian kernel has a $PD$ decomposition on any Abelian Lie group, and non-commutative settings where smooth, rapidly decaying functions yield $PD$ decomposability on homogeneous spaces such as non-compact symmetric spaces. These results establish a theoretical foundation for RKKS-based kernel learning on non-Euclidean data and broaden the applicability of kernel methods beyond PD kernels, with practical implications for learning on manifolds and groups.

Abstract

Kernel methods are powerful tools in machine learning. Classical kernel methods are based on positive-definite kernels, which map data spaces into reproducing kernel Hilbert spaces (RKHS). For non-Euclidean data spaces, positive-definite kernels are difficult to come by. In this case, we propose the use of reproducing kernel Krein space (RKKS) based methods, which require only kernels that admit a positive decomposition. We show that one does not need to access this decomposition in order to learn in RKKS. We then investigate the conditions under which a kernel is positively decomposable. We show that invariant kernels admit a positive decomposition on homogeneous spaces under tractable regularity assumptions. This makes them much easier to construct than positive-definite kernels, providing a route for learning with kernels for non-Euclidean data. By the same token, this provides theoretical foundations for RKKS-based methods in general.

Figures (3)

• Figure 1: The Krein SVM algorithm bonnet-loosli_learning_2016 applied on the hyperbolic plane $\mathbb H^2$, with the geodesic Gaussian kernel $k = \exp(-\lambda d(\cdot ,\cdot )^2)$. The data is sampled from a Riemannian Gaussian distribution Said2018HOS2022Said2023 centered at the origin of the Poincare disc and is split into two classes according to geodesic decision boundaries (dotted curves in the figure). The number of sampled data points is 200, 200, and 500, respectively. The results of the classification are displayed in the Poincare disc model of $\mathbb H^2$. We will show in Corollary \ref{['symmetric_cor']} that the Gaussian kernel admits a PD decomposition on $\mathbb H^2$, justifying its use in this scenario.
• Figure 2: A geodesic in the hyperboloid model of the hyperbolic plane $\mathbb H^2$, obtained as the intersection of a plane with the hyperboloid. The corresponding geodesic on the Poincare disc model $\mathbb B^2$ of the hyperbolic plane is obtained by stereographic projection from the point $(0,0,-1)$.
• Figure 3: A visualisation of the projections to the double coset spaces $\pi: S^2\cong SO(3)/SO(2)\to SO(2)\backslash SO(3)/SO(2)$ and $\pi: \mathbb S_{++}^2\cong GL(2)/O(2)\to O(2)\backslash GL(2)/O(2)$. In each case, we have in red an orbit $HgH$. In the case of $S^2$, it is obtained by rotating $gSO(2)$ around a horizontal axis. In the case of $\mathbb S_{++}^2$, it is the set of matrices with eigenvalues equal to the ones of $gO(2)$, obtained as the intersection of the constant-trace plane and the constant-determinant surface. In each case, we have in dark blue a set transversal to such orbits, and homeomorphic to $H\backslash G/H$.

Theorems & Definitions (66)

• Definition 1
• Definition 2
• Theorem 3
• Example 4
• Example 5
• Definition 6
• Definition 7
• Remark 8
• Theorem 9
• Remark 10