Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Non-separable Covariance Kernels for Spatiotemporal Gaussian Processes based on a Hybrid Spectral Method and the Harmonic Oscillator

Dionissios T. Hristopulos

TL;DR

This work introduces a physics-based, hybrid spectral framework to construct non-separable spatiotemporal covariance kernels for Gaussian processes, grounded in the stochastic linear damped harmonic oscillator (LDHO). By embedding dispersion relations that couple spatial frequency to temporal dynamics, the method yields explicit, isotropic LDHO kernels across underdamped, critically damped, and overdamped regimes, and also extends to Ornstein–Uhlenbeck kernels. The approach ensures Bochner admissibility while enabling space–time interactions and oscillatory temporal correlations, with clear physical interpretation of hyperparameters (amplitude, damping, natural frequency, variance decay, and interaction strength). These kernels exhibit rich behavior, including hole effects, quasi-periodicity, and controllable space–time coupling, making them suitable for geophysical and environmental spatiotemporal data where physical processes drive the correlations. The results offer a foundation for scalable GP modeling, parameter estimation, and potential extensions to multivariate processes and manifolds, bridging physical models and data-driven inference in a principled way.

Abstract

Gaussian processes provide a flexible, non-parametric framework for the approximation of functions in high-dimensional spaces. The covariance kernel is the main engine of Gaussian processes, incorporating correlations that underpin the predictive distribution. For applications with spatiotemporal datasets, suitable kernels should model joint spatial and temporal dependence. Separable space-time covariance kernels offer simplicity and computational efficiency. However, non-separable kernels include space-time interactions that better capture observed correlations. Most non-separable kernels that admit explicit expressions are based on mathematical considerations (admissibility conditions) rather than first-principles derivations. We present a hybrid spectral approach for generating covariance kernels which is based on physical arguments. We use this approach to derive a new class of physically motivated, non-separable covariance kernels which have their roots in the stochastic, linear, damped, harmonic oscillator (LDHO). The new kernels incorporate functions with both monotonic and oscillatory decay of space-time correlations. The LDHO covariance kernels involve space-time interactions which are introduced by dispersion relations that modulate the oscillator coefficients. We derive explicit relations for the spatiotemporal covariance kernels in the three oscillator regimes (underdamping, critical damping, overdamping) and investigate their properties. We further illustrate the hybrid spectral method by deriving covariance kernels that are based on the Ornstein-Uhlenbeck model.

Non-separable Covariance Kernels for Spatiotemporal Gaussian Processes based on a Hybrid Spectral Method and the Harmonic Oscillator

TL;DR

This work introduces a physics-based, hybrid spectral framework to construct non-separable spatiotemporal covariance kernels for Gaussian processes, grounded in the stochastic linear damped harmonic oscillator (LDHO). By embedding dispersion relations that couple spatial frequency to temporal dynamics, the method yields explicit, isotropic LDHO kernels across underdamped, critically damped, and overdamped regimes, and also extends to Ornstein–Uhlenbeck kernels. The approach ensures Bochner admissibility while enabling space–time interactions and oscillatory temporal correlations, with clear physical interpretation of hyperparameters (amplitude, damping, natural frequency, variance decay, and interaction strength). These kernels exhibit rich behavior, including hole effects, quasi-periodicity, and controllable space–time coupling, making them suitable for geophysical and environmental spatiotemporal data where physical processes drive the correlations. The results offer a foundation for scalable GP modeling, parameter estimation, and potential extensions to multivariate processes and manifolds, bridging physical models and data-driven inference in a principled way.

Abstract

Gaussian processes provide a flexible, non-parametric framework for the approximation of functions in high-dimensional spaces. The covariance kernel is the main engine of Gaussian processes, incorporating correlations that underpin the predictive distribution. For applications with spatiotemporal datasets, suitable kernels should model joint spatial and temporal dependence. Separable space-time covariance kernels offer simplicity and computational efficiency. However, non-separable kernels include space-time interactions that better capture observed correlations. Most non-separable kernels that admit explicit expressions are based on mathematical considerations (admissibility conditions) rather than first-principles derivations. We present a hybrid spectral approach for generating covariance kernels which is based on physical arguments. We use this approach to derive a new class of physically motivated, non-separable covariance kernels which have their roots in the stochastic, linear, damped, harmonic oscillator (LDHO). The new kernels incorporate functions with both monotonic and oscillatory decay of space-time correlations. The LDHO covariance kernels involve space-time interactions which are introduced by dispersion relations that modulate the oscillator coefficients. We derive explicit relations for the spatiotemporal covariance kernels in the three oscillator regimes (underdamping, critical damping, overdamping) and investigate their properties. We further illustrate the hybrid spectral method by deriving covariance kernels that are based on the Ornstein-Uhlenbeck model.
Paper Structure (51 sections, 22 theorems, 158 equations, 11 figures)

This paper contains 51 sections, 22 theorems, 158 equations, 11 figures.

Table of Contents

  1. INTRODUCTION
  2. Methods and Procedures
  3. Notation and Definitions
  4. Gaussian processes and random fields
  5. Covariance kernels
  6. Covariance Kernel of Harmonic Oscillator Driven by White Noise
  7. Equation of motion
  8. Covariance equation of motion
  9. Covariance kernel
  10. Underdamping
  11. Overdamping
  12. Critical damping
  13. Hybrid Spectral Approach for Spatiotemporal Kernel Construction
  14. Hybrid Spectral Approach Applied to the Harmonic Oscillator
  15. Dispersion relations
  16. ...and 36 more sections

Key Result

Theorem 1

A function $C({\bf r},\tau)$ is an admissible covariance kernel for a stationary random field if and only if the Fourier transform $\widetilde{C}({\bf k},\omega)$ of $C({\bf r},\tau)$ exists, is non-negative, and its integral over $\mathbb{R}^d \times \mathbb{R}$ is finite Bochner59.

Figures (11)

  • Figure 1: Normalized $C(r,\tau)$ and isolevel contour lines (red online) in the underdamped regime, obtained from \ref{['eq:ldho-kernel-st-under']} using $\tilde{\omega}_{d}=3\pi/2$, $\tilde{\tau}_{c}=3$, $b=0.4$, $\epsilon=1$, and $d=2$.
  • Figure 2: Normalized $C(r,\tau)$ and isolevel contour lines (red online) in the overdamped regime, obtained from \ref{['eq:ldho-kernel-st-over']} using $\tilde{\omega}_{d}=\pi/10$, $\tilde{\tau}_{c}=0.8$, $b=0.4$, $\epsilon=8$, and $d=2$.
  • Figure 3: Normalized $C(r,\tau)$ and isolevel contours (red online) in the underdamped regime, obtained from \ref{['eq:ldho-kernel-st-under']} using $\tilde{\omega}_{d}=3\pi/2$, $\tilde{\tau}_{c}=3$, $b=0.4$, $\epsilon=3$, and $d=2$.
  • Figure 4: First nine time slices of the synthetic data corresponding to $z({\mathbf s}, t)$, where $t=1, \ldots, 9$.
  • Figure 5: Time series of the synthetic data corresponding to $z(s_{i}, s_{i}, t)$ where $s_{i}=7\,(i-1)+ 1$, and $i=1, 2, \ldots, 9$.
  • ...and 6 more figures

Theorems & Definitions (43)

  • Definition 1: Non-negative definite functions
  • Remark 1: Units
  • Definition 2: Fourier transforms
  • Theorem 1: Bochner's theorem
  • Definition 3: Radial functions
  • Definition 4: Marginal covariance kernels
  • Remark 2: RLC Oscillator
  • Corollary 1: Green's function equivalence of LDHO Covariance
  • Corollary 2: Spectral density from generative ODE
  • Remark 3: Admissibility of spectral density
  • ...and 33 more