Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gaussian Process Regression under Computational and Epistemic Misspecification

Daniel Sanz-Alonso, Ruiyi Yang

TL;DR

This paper develops a unified framework to analyze Gaussian process regression under simultaneous epistemic and computational misspecification, focusing on interpolation error when the employed kernel is a misspecified approximation of the true kernel. The authors bound the misspecified kriging error by decomposing it into a well-understood interpolation error plus an approximation error controlled via a bridge function $f_N$ in the misspecified RKHS, and a stability term. They instantiate the framework in four families of kernel misspecifications: Karhunen-Loève truncations, wavelet-based multiscale kernels, and finite element representations, along with a Matérn-parameter misspecification example, deriving explicit $L^2$ and $L^{\\infty}$ rates that depend on the fill distance $h_{n,\\Omega}$, the complexity parameter $N$, and regularity indices. The results recover known bounds in the noiseless Matérn setting and provide new rates and design guidelines for scalable GP interpolation under practical kernel-approximation strategies, informing how to balance approximation complexity against the number of observations. Together, these insights offer theoretical guidance for constructing scalable GP surrogates with provable accuracy in large-data settings and for choosing kernel-complexity targets to meet a desired error tolerance.

Abstract

Gaussian process regression is a classical kernel method for function estimation and data interpolation. In large data applications, computational costs can be reduced using low-rank or sparse approximations of the kernel. This paper investigates the effect of such kernel approximations on the interpolation error. We introduce a unified framework to analyze Gaussian process regression under important classes of computational misspecification: Karhunen-Loève expansions that result in low-rank kernel approximations, multiscale wavelet expansions that induce sparsity in the covariance matrix, and finite element representations that induce sparsity in the precision matrix. Our theory also accounts for epistemic misspecification in the choice of kernel parameters.

Gaussian Process Regression under Computational and Epistemic Misspecification

TL;DR

This paper develops a unified framework to analyze Gaussian process regression under simultaneous epistemic and computational misspecification, focusing on interpolation error when the employed kernel is a misspecified approximation of the true kernel. The authors bound the misspecified kriging error by decomposing it into a well-understood interpolation error plus an approximation error controlled via a bridge function in the misspecified RKHS, and a stability term. They instantiate the framework in four families of kernel misspecifications: Karhunen-Loève truncations, wavelet-based multiscale kernels, and finite element representations, along with a Matérn-parameter misspecification example, deriving explicit and rates that depend on the fill distance , the complexity parameter , and regularity indices. The results recover known bounds in the noiseless Matérn setting and provide new rates and design guidelines for scalable GP interpolation under practical kernel-approximation strategies, informing how to balance approximation complexity against the number of observations. Together, these insights offer theoretical guidance for constructing scalable GP surrogates with provable accuracy in large-data settings and for choosing kernel-complexity targets to meet a desired error tolerance.

Abstract

Gaussian process regression is a classical kernel method for function estimation and data interpolation. In large data applications, computational costs can be reduced using low-rank or sparse approximations of the kernel. This paper investigates the effect of such kernel approximations on the interpolation error. We introduce a unified framework to analyze Gaussian process regression under important classes of computational misspecification: Karhunen-Loève expansions that result in low-rank kernel approximations, multiscale wavelet expansions that induce sparsity in the covariance matrix, and finite element representations that induce sparsity in the precision matrix. Our theory also accounts for epistemic misspecification in the choice of kernel parameters.
Paper Structure (31 sections, 25 theorems, 136 equations, 1 table)

This paper contains 31 sections, 25 theorems, 136 equations, 1 table.

Table of Contents

  1. Introduction
  2. Comparison to related work
  3. Outline
  4. Notation
  5. Preliminaries
  6. Problem formulation
  7. Experimental region and design points
  8. Unified theoretical framework
  9. Main Result
  10. Proof of the Main Result
  11. (Part I)
  12. (Part II)
  13. Epistemic and computational misspecification
  14. Epistemic misspecification in the Matérn model
  15. Background
  16. ...and 16 more sections

Key Result

theorem 2

Let $\Phi_N:\mathcal{D}\times \mathcal{D}\rightarrow \mathbb{R}$ be a covariance function whose RKHS satisfies $\mathcal{H}_{\Phi_{\scaleto{N}{2.5pt}}} \subset H^{s_{\scaleto{N}{2.5pt}}} (\mathcal{D})$ for $s_N>\frac{d}{2}$ with equivalent norms: there exists $A_N>1$ such that $A_N^{-1}\|g\|_{\math If $\lambda=0$, the constant $C_{s_{\scaleto{0}{2.5pt}},s_{\scaleto{N}{2.5pt}},d,\Omega}$ can be im

Theorems & Definitions (52)

  • theorem 2
  • Remark 3
  • Remark 4
  • Remark 5
  • Remark 6
  • Lemma 7: arcangeli2012extension and narcowich2005sobolev
  • Lemma 8: wendland2005approximate
  • Lemma 9
  • Lemma 10
  • proof : Proof of Lemma \ref{['lemma:approx interpolation error']}
  • ...and 42 more