Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces II: non-compact symmetric spaces

Iskander Azangulov, Andrei Smolensky, Alexander Terenin, Viacheslav Borovitskiy

Abstract

Gaussian processes are arguably the most important class of spatiotemporal models within machine learning. They encode prior information about the modeled function and can be used for exact or approximate Bayesian learning. In many applications, particularly in physical sciences and engineering, but also in areas such as geostatistics and neuroscience, invariance to symmetries is one of the most fundamental forms of prior information one can consider. The invariance of a Gaussian process' covariance to such symmetries gives rise to the most natural generalization of the concept of stationarity to such spaces. In this work, we develop constructive and practical techniques for building stationary Gaussian processes on a very large class of non-Euclidean spaces arising in the context of symmetries. Our techniques make it possible to (i) calculate covariance kernels and (ii) sample from prior and posterior Gaussian processes defined on such spaces, both in a practical manner. This work is split into two parts, each involving different technical considerations: part I studies compact spaces, while part II studies non-compact spaces possessing certain structure. Our contributions make the non-Euclidean Gaussian process models we study compatible with well-understood computational techniques available in standard Gaussian process software packages, thereby making them accessible to practitioners.

Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces II: non-compact symmetric spaces

Abstract

Gaussian processes are arguably the most important class of spatiotemporal models within machine learning. They encode prior information about the modeled function and can be used for exact or approximate Bayesian learning. In many applications, particularly in physical sciences and engineering, but also in areas such as geostatistics and neuroscience, invariance to symmetries is one of the most fundamental forms of prior information one can consider. The invariance of a Gaussian process' covariance to such symmetries gives rise to the most natural generalization of the concept of stationarity to such spaces. In this work, we develop constructive and practical techniques for building stationary Gaussian processes on a very large class of non-Euclidean spaces arising in the context of symmetries. Our techniques make it possible to (i) calculate covariance kernels and (ii) sample from prior and posterior Gaussian processes defined on such spaces, both in a practical manner. This work is split into two parts, each involving different technical considerations: part I studies compact spaces, while part II studies non-compact spaces possessing certain structure. Our contributions make the non-Euclidean Gaussian process models we study compatible with well-understood computational techniques available in standard Gaussian process software packages, thereby making them accessible to practitioners.
Paper Structure (30 sections, 32 theorems, 143 equations, 10 figures, 1 table, 2 algorithms)

This paper contains 30 sections, 32 theorems, 143 equations, 10 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Notation
  3. Gaussian Processes and Gaussian Process Regression
  4. Gaussian Processes That are Stationary Under Group Action
  5. Stationary Gaussian Processes on Symmetric Spaces
  6. Computational Techniques for Stationary Gaussian Processes
  7. Enumerating Representations of Symmetric Spaces
  8. Pointwise Kernel Evaluation
  9. Efficient Sampling
  10. Heat and Matérn Kernels
  11. Choice of Ordinary vs. Shifted Laplace--Beltrami Operator
  12. Stationarity and Spectral Densities
  13. General Computation Techniques
  14. Examples
  15. Hyperbolic Space
  16. ...and 15 more sections

Key Result

theorem 1

Let $X = G/H$ be a non-compact symmetric space, where $G$ is a Lie group of type I.We do not dwell on the technical definition of a type I group, which involves $C^*$-algebraic considerations and implies that all irreducible unitary representations are uniquely characterized by characters, which is for a function $\Bbbk : G \-> \R$, where $g_1 \mathbin{\vcenter{\hbox{$\bullet$}}} H$ and $g_2 \mat

Figures (10)

  • Figure 1: Posterior on the manifold $\mathop{\mathrm{SPD}}\nolimits(d)$ of symmetric positive definite $d \x d$ matrices, equipped with the affine-invariant metric. Here, $d=2$ and $\mathop{\mathrm{SPD}}\nolimits(d)$ is represented as a cone: see \ref{['sec:spd']}. The top and the bottom rows correspond to two cross-sections thereof.
  • Figure 2: Values of the heat kernel $k(\bullet,\.)$ on $\mathbb{H}_2$ and $\mathop{\mathrm{SPD}}\nolimits(2)$.
  • Figure 3: Samples from a Gaussian process with heat kernel covariance on $\mathbb{H}_2$ and $\mathop{\mathrm{SPD}}\nolimits(2)$.
  • Figure 4: Samples from Matérn Gaussian processes on $\mathbb{H}_2$, with varying smoothness.
  • Figure 5: Gaussian process regression on hyperbolic space $\mathbb{H}_2$ driven by the heat kernel.
  • ...and 5 more figures

Theorems & Definitions (76)

  • proof : Proof sketch
  • theorem 1
  • proof
  • proposition 1
  • proof
  • proposition 2
  • proof
  • proof
  • proposition 3
  • proof
  • ...and 66 more