Regular Fourier Features for Nonstationary Gaussian Processes

TL;DR

This work discretizes the spectral representation directly, preserving the correlation structure among spectral weights without requiring probability assumptions, and yields an efficient low-rank approximation that is positive semi-definite by construction.

Abstract

Simulating a Gaussian process requires sampling from a high-dimensional Gaussian distribution, which scales cubically with the number of sample locations. Spectral methods address this challenge by exploiting the Fourier representation, treating the spectral density as a probability distribution for Monte Carlo approximation. Although this probabilistic interpretation works for stationary processes, it is overly restrictive for the nonstationary case, where spectral densities are generally not probability measures. We propose regular Fourier features for harmonizable processes that avoid this limitation. Our method discretizes the spectral representation directly, preserving the correlation structure among spectral weights without requiring probability assumptions. Under a finite spectral support assumption, this yields an efficient low-rank approximation that is positive semi-definite by construction. When the spectral density is unknown, the framework extends naturally to kernel learning from data. We demonstrate the method on locally stationary kernels and on harmonizable mixture kernels with complex-valued spectral densities.

Figures (5)

• Figure 1: Regular Fourier features approximate the stochastic process $Z(x)$ as a sum of complex exponentials weighted by correlated spectral increments $\mathrm{d}\Gamma(\omega_k)$. The spectral process (top) is discretized at frequencies $\omega_1, \ldots, \omega_m$. Each harmonic (bottom) shows the contribution $\mathrm{d}\Gamma(\omega_k) \exp(\mathrm{i}\omega_k x)$ in the complex plane.
• Figure 2: Low-rank approximation of the locally stationary kernel showing the sampled spectral density, true kernel, approximation, and absolute error.
• Figure 3: Harmonizable mixture kernel approximation with complex-valued spectral density. Top row: real, imaginary, and absolute values of $s(\omega, \omega')$. Bottom row: true kernel, approximation, and absolute error.
• Figure 4: Ablation studies on the locally stationary kernel showing relative error versus (top) number of features $m$ with fixed $\omega_{m}=5$, and (bottom) cutoff frequency $\omega_{m}$ with fixed $m=100$, across different problem scales $n$.
• Figure 5: Posterior predictions on synthetic data from the Silverman locally stationary kernel. Left: true posterior. Center: our method compared to true (dashed). Right: RBF baseline compared to true (dashed).