Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Constructing Gaussian Processes via Samplets

Marcel Neugebauer

TL;DR

This master's thesis proposes a Samplet-based approach to efficiently construct and train the Gaussian Processes, reducing the cubic computational complexity to a log-linear scale and facilitates optimal regression while maintaining efficient performance.

Abstract

Gaussian Processes face two primary challenges: constructing models for large datasets and selecting the optimal model. This master's thesis tackles these challenges in the low-dimensional case. We examine recent convergence results to identify models with optimal convergence rates and pinpoint essential parameters. Utilizing this model, we propose a Samplet-based approach to efficiently construct and train the Gaussian Processes, reducing the cubic computational complexity to a log-linear scale. This method facilitates optimal regression while maintaining efficient performance.

Constructing Gaussian Processes via Samplets

TL;DR

This master's thesis proposes a Samplet-based approach to efficiently construct and train the Gaussian Processes, reducing the cubic computational complexity to a log-linear scale and facilitates optimal regression while maintaining efficient performance.

Abstract

Gaussian Processes face two primary challenges: constructing models for large datasets and selecting the optimal model. This master's thesis tackles these challenges in the low-dimensional case. We examine recent convergence results to identify models with optimal convergence rates and pinpoint essential parameters. Utilizing this model, we propose a Samplet-based approach to efficiently construct and train the Gaussian Processes, reducing the cubic computational complexity to a log-linear scale. This method facilitates optimal regression while maintaining efficient performance.

Paper Structure

This paper contains 41 sections, 37 theorems, 284 equations, 24 figures, 2 tables, 11 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Gaussian Processes
  4. Reproducing Kernel Hilbert Spaces
  5. Model Selection
  6. Mean function
  7. Kernel function
  8. Fitting Hyperparameters
  9. Bayesian Optimization
  10. Convergence of Gaussian Process Means
  11. Sobolev spaces
  12. Experimental setting
  13. Experimental design
  14. Gaussian Process
  15. Convergence guarantees
  16. ...and 26 more sections

Key Result

Proposition 2.1.5

Given $f \sim \mathcal{G}\mathcal{P} (m,k)$ and observations $\mathcal{D}_N = \{ (\boldsymbol{x}_1, y_1), ..., (\boldsymbol{x}_N, y_N) \}$. Then for the posterior function holds $f \, | \, \mathcal{D}_N \sim \mathcal{G}\mathcal{P} (m',k')$, where for $\boldsymbol{k}(\boldsymbol{x}) = [k(\boldsymbol{x},\boldsymbol{x}_1),...,k(\boldsymbol{x},\boldsymbol{x}_N)]$, $\boldsymbol{m} = [m(\boldsymbol{x}

Figures (24)

  • Figure 1: Visualization of different Gaussian Processes by plotting mean function, confidence intervals and samples. The top left panel shows $\mathcal{G}\mathcal{P} (0,k)$ with kernel given by equation (\ref{['eq. 2.1.1']}). The three other panels consider in addition noisy observations and visualize the associated posterior process $\mathcal{G}\mathcal{P} (m',k')$ for $\sigma^2 = 1$.
  • Figure 2: Comparison of the GP in the bottom right panel of Figure \ref{['fig. 1']} with the function $4x\sin(10x)$, from which the noisy observations were sampled.
  • Figure 3: Samples of three Gaussian Processes. The middle plot alters the mean function, while the right plot modifies the kernel, in comparison to the left plot.
  • Figure 4: Visualization of posterior Gaussian Processes. The top left panel shows the posterior Process of $\mathcal{G}\mathcal{P} (0,k)$, the top right panel of $\mathcal{G}\mathcal{P} (m,k)$ with parameters from Example \ref{['2.3.1']} and the bottom panel of $\mathcal{G}\mathcal{P} (0,k_{1/2,1})$ with Matérn $1/2$ kernel. Observations were sampled from the same function as in Figure \ref{['fig. 2']}, but without noise.
  • Figure 5: Samples of Gaussian Processes $\mathcal{GP}(0,k)$.
  • ...and 19 more figures

Theorems & Definitions (116)

  • Definition 2.1.1
  • Definition 2.1.2
  • Remark 2.1.3
  • Example 2.1.4
  • Proposition 2.1.5
  • proof
  • Example 2.1.6
  • Definition 2.2.1
  • Proposition 2.2.2
  • proof
  • ...and 106 more