Intrinsic Gaussian Processes on Manifolds and Their Accelerations by Symmetry

TL;DR

An intrinsic approach of constructing the Gaussian process on general manifolds such as orthogonal groups, unitary groups, Stiefel manifolds and Grassmannian manifolds is proposed and the ball algorithm and the strip algorithm for manifolds with extra symmetries are proposed.

Abstract

Amidst the growing interest in nonparametric regression, we address a significant challenge in Gaussian processes(GP) applied to manifold-based predictors. Existing methods primarily focus on low dimensional constrained domains for heat kernel estimation, limiting their effectiveness in higher-dimensional manifolds. Our research proposes an intrinsic approach for constructing GP on general manifolds such as orthogonal groups, unitary groups, Stiefel manifolds and Grassmannian manifolds. Our methodology estimates the heat kernel by simulating Brownian motion sample paths using the exponential map, ensuring independence from the manifold's embedding. The introduction of our strip algorithm, tailored for manifolds with extra symmetries, and the ball algorithm, designed for arbitrary manifolds, constitutes our significant contribution. Both algorithms are rigorously substantiated through theoretical proofs and numerical testing, with the strip algorithm showcasing remarkable efficiency gains over traditional methods. This intrinsic approach delivers several key advantages, including applicability to high dimensional manifolds, eliminating the requirement for global parametrization or embedding. We demonstrate its practicality through regression case studies (torus knots and eight dimensional projective spaces) and by developing binary classifiers for real world datasets (gorilla skulls planar images and diffusion tensor images). These classifiers outperform traditional methods, particularly in limited data scenarios.

Figures (3)

• Figure 1: Illustrative examples of the Brownian motion on manifold $M$. Five independent BM paths from time $0$ to $t$ represented by the solid coloured lines in both \ref{['fig:ball']} and \ref{['fig:strip']}. $x_0$ is the starting point of BM paths. In \ref{['fig:ball']} only one sample path (red) reaches $D$ at time $t$, so the transition probability is $1/5$. In \ref{['fig:strip']} there are two paths (red and blue) reach the strip at time $t$ so the transition probability is $2/5$.
• Figure 2: Comparison of the estimate of heat kernel on $\mathbb{R}^1$, $\mathbb{R}^2$ and $\mathbb{R}^3$ using strip and ball algorithms. The number of Brownian motion sample paths are 20000 for all three cases. The same window size and strip width are used. The true kernel values are plotted in dotted blue line while the estimations from ball algorithm are in green dot-dash line and strip method are in red dash line.
• Figure 3: Comparison of the truth and prediction using extrinsic GP with RBF kernel embedded in $R^3$ and SiGP. The true value of the regression function is plotted in colour.

Theorems & Definitions (22)

• Theorem 2.1
• Theorem 3.1
• Theorem 3.2
• proof
• Lemma 1.1
• proof
• Remark 1.2
• Proposition 2.1
• proof
• Lemma 2.2