Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal uncertainty bounds for multivariate kernel regression under bounded noise: A Gaussian process-based dual function

Amon Lahr, Anna Scampicchio, Johannes Köhler, Melanie N. Zeilinger

Abstract

Non-conservative uncertainty bounds are essential for making reliable predictions about latent functions from noisy data--and thus, a key enabler for safe learning-based control. In this domain, kernel methods such as Gaussian process regression are established techniques, thanks to their inherent uncertainty quantification mechanism. Still, existing bounds either pose strong assumptions on the underlying noise distribution, are conservative, do not scale well in the multi-output case, or are difficult to integrate into downstream tasks. This paper addresses these limitations by presenting a tight, distribution-free bound for multi-output kernel-based estimates. It is obtained through an unconstrained, duality-based formulation, which shares the same structure of classic Gaussian process confidence bounds and can thus be straightforwardly integrated into downstream optimization pipelines. We show that the proposed bound generalizes many existing results and illustrate its application using an example inspired by quadrotor dynamics learning.

Optimal uncertainty bounds for multivariate kernel regression under bounded noise: A Gaussian process-based dual function

Abstract

Non-conservative uncertainty bounds are essential for making reliable predictions about latent functions from noisy data--and thus, a key enabler for safe learning-based control. In this domain, kernel methods such as Gaussian process regression are established techniques, thanks to their inherent uncertainty quantification mechanism. Still, existing bounds either pose strong assumptions on the underlying noise distribution, are conservative, do not scale well in the multi-output case, or are difficult to integrate into downstream tasks. This paper addresses these limitations by presenting a tight, distribution-free bound for multi-output kernel-based estimates. It is obtained through an unconstrained, duality-based formulation, which shares the same structure of classic Gaussian process confidence bounds and can thus be straightforwardly integrated into downstream optimization pipelines. We show that the proposed bound generalizes many existing results and illustrate its application using an example inspired by quadrotor dynamics learning.
Paper Structure (15 sections, 4 theorems, 23 equations, 2 figures, 1 table)

This paper contains 15 sections, 4 theorems, 23 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem setting
  3. Main result
  4. Discussion
  5. Optimal deterministic bounds (optimization-based)
  6. Suboptimal deterministic bounds (closed-form)
  7. Finite-dimensional hypothesis spaces
  8. Probabilistic bounds
  9. Application example
  10. Conclusions
  11. Auxiliary results
  12. Proof of \ref{['thm:optimal_bound']}
  13. Proof of statements i) and ii)
  14. Proof of statement iii)
  15. Proof of \ref{['eq:relaxed_bound']}

Key Result

Theorem 1

Let ass:bounded_noiseass:rkhs_norm_f hold and define with Then, for any $h \in \mathbb{R}^{n_f}$ and any $x_{N+1} \in \mathbb{R}^{n_x}$,

Figures (2)

  • Figure 1: Illustrative example of proposed uncertainty bound. The top plot shows the optimal uncertainty bounds (solid), as well as the corresponding dual functions (shaded), evaluated for the optimal dual (noise) parameters $\sigma^\star$ at the test point $x_{N+1} = 1.5$ (dotted red). The bottom two plots show the corresponding optimal value of the dual (noise) parameters $\sigma^\star$ for all test points.
  • Figure 2: Proposed multivariate uncertainty bound for quadrotor example with $n_{\mathrm{data}} = 10$ training points. The latent function (dashed black) is tightly bounded by the optimal uncertainty bounds evaluated for both output dimensions (\ref{['thm:optimal_bound']}); the multivariate ellipsoidal tube is generated using $\sigma$-values corresponding to the optimal upper bound in $x$-direction (\ref{['thm:suboptimal_bound_multivariate']}). The (projected) data points and (projected) uncertainty bounds are shown in red and gray, respectively.

Theorems & Definitions (8)

  • Theorem 1
  • Corollary 1
  • proof
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • proof