Error Bounds For Gaussian Process Regression Under Bounded Support Noise With Applications To Safety Certification

TL;DR

This work tackles rigorous uncertainty quantification for Gaussian Process Regression under bounded noise, addressing a gap where traditional GP bounds are overly conservative for safety-critical applications. By decomposing the error into a mean-prediction term and a noise-perturbation term and applying concentration inequalities in an RKHS setting, the authors derive both probabilistic and deterministic bounds that are tighter than prior results. They further extend these bounds to Deep Kernel Learning, leveraging informed kernels to reduce predictive uncertainty, and demonstrate substantial improvements in safety certification through stochastic barrier functions. The practical impact is improved reliability and confidence in data-driven safety guarantees for unknown dynamical systems. These advances enable more efficient and robust safety analysis in control and robotics settings with bounded, non-Gaussian noise.

Abstract

Gaussian Process Regression (GPR) is a powerful and elegant method for learning complex functions from noisy data with a wide range of applications, including in safety-critical domains. Such applications have two key features: (i) they require rigorous error quantification, and (ii) the noise is often bounded and non-Gaussian due to, e.g., physical constraints. While error bounds for applying GPR in the presence of non-Gaussian noise exist, they tend to be overly restrictive and conservative in practice. In this paper, we provide novel error bounds for GPR under bounded support noise. Specifically, by relying on concentration inequalities and assuming that the latent function has low complexity in the reproducing kernel Hilbert space (RKHS) corresponding to the GP kernel, we derive both probabilistic and deterministic bounds on the error of the GPR. We show that these errors are substantially tighter than existing state-of-the-art bounds and are particularly well-suited for GPR with neural network kernels, i.e., Deep Kernel Learning (DKL). Furthermore, motivated by applications in safety-critical domains, we illustrate how these bounds can be combined with stochastic barrier functions to successfully quantify the safety probability of an unknown dynamical system from finite data. We validate the efficacy of our approach through several benchmarks and comparisons against existing bounds. The results show that our bounds are consistently smaller, and that DKLs can produce error bounds tighter than sample noise, significantly improving the safety probability of control systems.

Figures (5)

• Figure 1: Predictive mean and error bounds when learning from 20 samples of $y = x\sin(x) + v$ with $|v| < 0.5$. In (a) we plot the comparison of deterministic error bounds and mean predictions, and in (b) we show the probabilistic bounds that hold with 95% probability. We set $\sigma_n = 0.1$ for our predictions, $\sigma_n = 0.5$ for Hashimoto et al. as per Lemma \ref{['lemma:2']}, and $\sigma_n^2 = 1 + 2/20$ for Chowdhury et al. as per Lemma \ref{['lemma:1']}. Note bounds for abbasi2013online are nearly identical to Chowdhury et al. in this example, see Appendix \ref{['sec:appendix_prob']}.
• Figure 2: Predictive mean and deterministic error bounds when learning from 20 samples of $y = x\sin(x) + v$ with $|v| < 0.5$ as $\sigma_n$ varies. We plot the mean and bounds from Lemma \ref{['lemma:2']} in red, with our mean and bounds in green. In all cases our bounds remain valid, but demonstrate optimal performance when $\sigma_n \in [\sigma_v/10, \sigma_v/5]$.
• Figure 3: Trends of error bounds with increasing data used for the posterior prediction for a 2D system when $|v|\leq 0.1$ using Left: the squared exponential kernel and Right: DKL. In (a) and (c) we compare our deterministic error bound (green) with results from Lemma \ref{['lemma:2']} (red). In (b) and (d) we compare our probabilistic error bound (green) to results from Lemma \ref{['lemma:1']} (red) and abbasi2013online (orange) with $\delta = 0.05$. In (c) and (f) we compare our deterministic and probabilistic bounds with $\delta\in [0.01, 0.5]$.
• Figure 4: Trends of our error bounds with increasing data used for the posterior prediction for a 2D system when $|v|\leq 0.1$ using (a) the SE kernel and (b) DKL. We compare our deterministic (Theorem \ref{['my_det_theorem']}) and probabilistic (Theorem \ref{['my_prob_theorem']}) bounds with $\delta\in [0.01, 0.5]$.
• Figure 5: Predictive mean and probabilistic error bounds for $\delta = 0.05$ when learning from 20 samples of $y = x\sin(x) + v$ with $|v| < 0.5$. In (a) we plot the comparison with abbasi2013online, and in (b) we show the comparison with Chowdhury We set $\sigma_n = 0.1$ for our predictions and $\sigma_n^2 = 1 + 2/20$ for Chowdhury as per Lemma \ref{['lemma:1']} and for abbasi2013online

Theorems & Definitions (18)

• Lemma 1: Chowdhury
• Lemma 2: hashimoto2022learning
• Lemma 3
• Theorem 1: Bounded Support Probabilistic RKHS Error
• Corollary 1: Uniform Error Bounds
• Theorem 2: Bounded Support Deterministic RKHS Error
• proof
• Remark 1
• Remark 2
• Remark 3