Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explaining and Connecting Kriging with Gaussian Process Regression

Marius Marinescu

TL;DR

This work addresses the common but imprecise claim that Kriging and Gaussian Process Regression are the same by deriving Kriging from first principles and mapping its three main variants to corresponding GPR setups. It provides a unified mathematical treatment, clarifies how Simple, Ordinary, and Universal Kriging relate to known/unknown mean and to GPR with and without mean priors, and discusses how kernel estimation differs between variogram-based Kriging and marginal-likelihood-based GPR. The contributions include a historical introduction, a variant-by-variant comparison, a common framework bridging linear regression and probabilistic prediction, and an analysis of when the two methods yield identical predictions and uncertainties. The findings illuminate the exact conditions under which Kriging and GPR coincide and highlight practical implications for cross-disciplinary application and knowledge transfer in geostatistics and machine learning.

Abstract

Kriging and Gaussian Process Regression are statistical methods that allow predicting the outcome of a random process or a random field by using a sample of correlated observations. In other words, the random process or random field is partially observed, and by using a sample a prediction is made, pointwise or as a whole, where the latter can be thought as a reconstruction. In addition, the techniques permit to give a measure of uncertainty of the prediction. The methods have different origins. Kriging comes from geostatistics, a field which started to develop around 1950 oriented to mining valuation problems, whereas Gaussian Process Regression has gained popularity in the area of machine learning in the last decade of the previous century. In the literature, the methods are usually presented as being the same technique. However, beyond this affirmation, the techniques have yet not been compared on a thorough mathematical basis and neither explained why and under which conditions this affirmation holds. Furthermore, Kriging has many variants and this affirmation should be precised. In this paper, this gap is filled. It is shown, step by step how both methods are deduced from the first principles -- with a major focus on Kriging, the mathematical connection between them, and which Kriging variant corresponds to which Gaussian Process Regression set up. The three most widely used versions of Kriging are considered: Simple Kriging, Ordinary Kriging and Universal Kriging. It is found, that despite their closeness, the techniques are different in their approach and assumptions, in a similar way the Least Square method, the Best Linear Unbiased Estimator method and the Likelihood method in regression do. I hope this work deepen the understanding of the relation between Kriging and Gaussian Process Regression, and serves as a cohesive introductory resource for researchers.

Explaining and Connecting Kriging with Gaussian Process Regression

TL;DR

This work addresses the common but imprecise claim that Kriging and Gaussian Process Regression are the same by deriving Kriging from first principles and mapping its three main variants to corresponding GPR setups. It provides a unified mathematical treatment, clarifies how Simple, Ordinary, and Universal Kriging relate to known/unknown mean and to GPR with and without mean priors, and discusses how kernel estimation differs between variogram-based Kriging and marginal-likelihood-based GPR. The contributions include a historical introduction, a variant-by-variant comparison, a common framework bridging linear regression and probabilistic prediction, and an analysis of when the two methods yield identical predictions and uncertainties. The findings illuminate the exact conditions under which Kriging and GPR coincide and highlight practical implications for cross-disciplinary application and knowledge transfer in geostatistics and machine learning.

Abstract

Kriging and Gaussian Process Regression are statistical methods that allow predicting the outcome of a random process or a random field by using a sample of correlated observations. In other words, the random process or random field is partially observed, and by using a sample a prediction is made, pointwise or as a whole, where the latter can be thought as a reconstruction. In addition, the techniques permit to give a measure of uncertainty of the prediction. The methods have different origins. Kriging comes from geostatistics, a field which started to develop around 1950 oriented to mining valuation problems, whereas Gaussian Process Regression has gained popularity in the area of machine learning in the last decade of the previous century. In the literature, the methods are usually presented as being the same technique. However, beyond this affirmation, the techniques have yet not been compared on a thorough mathematical basis and neither explained why and under which conditions this affirmation holds. Furthermore, Kriging has many variants and this affirmation should be precised. In this paper, this gap is filled. It is shown, step by step how both methods are deduced from the first principles -- with a major focus on Kriging, the mathematical connection between them, and which Kriging variant corresponds to which Gaussian Process Regression set up. The three most widely used versions of Kriging are considered: Simple Kriging, Ordinary Kriging and Universal Kriging. It is found, that despite their closeness, the techniques are different in their approach and assumptions, in a similar way the Least Square method, the Best Linear Unbiased Estimator method and the Likelihood method in regression do. I hope this work deepen the understanding of the relation between Kriging and Gaussian Process Regression, and serves as a cohesive introductory resource for researchers.
Paper Structure (17 sections, 53 equations, 1 figure, 1 table)

This paper contains 17 sections, 53 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Kriging historical introduction
  3. GPR historical introduction
  4. Mathematical formulation of Kriging
  5. Simple Kriging
  6. Ordinary Kriging
  7. Universal Kriging
  8. The bridge
  9. Mathematical formulation of GPR
  10. GPR with unknown mean
  11. Connections of the methods
  12. Unknown kernel
  13. Conclusion
  14. Equivalence of SK with zero-mean simplification
  15. Equivalency of OK with SK + GLS for the mean
  16. ...and 2 more sections

Figures (1)

  • Figure 1: D. Krige, observing the data, hypothesized that $E[\bar{Z} \mid Z_p=z_p]=z_p$ holds (dotted line), but the equality in the converse conditional expectation does not hold (continuous line). The observed under/over valuation of panels, reflected in Eqs. \ref{['eq:under']} and \ref{['eq:over']}, implies that $1=\rho\frac{\sigma_\text{\tiny X}^2}{\sigma_Y^2}$ which makes $\beta = \frac{\sigma_\text{\tiny Y}^2}{\sigma_X^2}$, and that $\beta <1$.