Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces I: the compact case

TL;DR

The techniques make it possible to calculate covariance kernels and sample from prior and posterior Gaussian processes defined on such spaces, both in a practical manner, and make the non-Euclidean Gaussian process models studied compatible with well-understood computational techniques available in standardGaussian process software packages, thereby making them accessible to practitioners.

Abstract

Gaussian processes are arguably the most important class of spatiotemporal models within machine learning. They encode prior information about the modeled function and can be used for exact or approximate Bayesian learning. In many applications, particularly in physical sciences and engineering, but also in areas such as geostatistics and neuroscience, invariance to symmetries is one of the most fundamental forms of prior information one can consider. The invariance of a Gaussian process' covariance to such symmetries gives rise to the most natural generalization of the concept of stationarity to such spaces. In this work, we develop constructive and practical techniques for building stationary Gaussian processes on a very large class of non-Euclidean spaces arising in the context of symmetries. Our techniques make it possible to (i) calculate covariance kernels and (ii) sample from prior and posterior Gaussian processes defined on such spaces, both in a practical manner. This work is split into two parts, each involving different technical considerations: part I studies compact spaces, while part II studies non-compact spaces possessing certain structure. Our contributions make the non-Euclidean Gaussian process models we study compatible with well-understood computational techniques available in standard Gaussian process software packages, thereby making them accessible to practitioners.

Figures (9)

• Figure 1: We illustrate a regression problem on a real projective plane $\mathop{\mathrm{RP}}\nolimits_2$, with a Matérn-3/2 kernel in the sense of \ref{['sec:heat_matern']}. Here and below we use the Boy's surface immersion of $\mathop{\mathrm{RP}}\nolimits_2$ into $\mathbb{R}^3$bryant1988, which is non-isometric. Regression is applied to model the function $y(x) = \sum_{j=1}^5 \cos(d_{\mathop{\mathrm{RP}}\nolimits_2}(x, \xi_j))$ where $\xi_j$ are uniform samples on $\mathop{\mathrm{RP}}\nolimits_2$. We show the resulting posterior mean and standard deviation, and one posterior sample.
• Figure 2: Illustration of the Matérn-3/2 kernel of \ref{['sec:heat_matern']}, and three random samples from its respective prior Gaussian process, on the circle group $\mathbb{S}_1$, which acts on itself by rotation.
• Figure 3: Illustration of the Matérn-3/2 kernel of \ref{['sec:heat_matern']}, and samples from its corresponding prior Gaussian process, on the sphere $G/H = \mathop{\mathrm{SO}}\nolimits(3)/\mathop{\mathrm{SO}}\nolimits(2) = \mathbb{S}_2$.
• Figure 4: Values of the heat kernel $k(\bullet,\.)$ on $\mathbb{T}^2$, $\mathop{\mathrm{RP}}\nolimits_2$, and $\mathbb{S}_2$. The torus $\mathbb{T}^2$ is equipped with the flat metric, so the depicted embedding is not isometric.
• Figure 5: Samples from a Gaussian process with heat kernel covariance, on $\mathbb{T}^2$, $\mathop{\mathrm{RP}}\nolimits_2$, and $\mathbb{S}_2$.

Theorems & Definitions (75)

• theorem 1
• proof
• theorem 2
• proof
• corollary 1
• proof
• theorem 3
• proof
• theorem 4
• proof