Distributionally Robust Gaussian Process Regression and Bayesian Inverse Problems

TL;DR

This work develops a distributionally robust optimization framework for infinite-dimensional Gaussian models, covering Gaussian process regression and linear inverse problems. It formulates a Wasserstein ambiguity set around a nominal Gaussian model with an RKHS-based ground cost to constrain perturbations and path roughness, proving strong duality and the existence of a unique Nash equilibrium for small ambiguity. The worst-case model remains Gaussian and the robust predictor is affine in the data, with finite-dimensional truncations yielding a computable solution via Frank-Wolfe iterations. Numerically, adversarial perturbations amplify uncertainty where data are scarce, while interacting with prior smoothness and forward-model complexity, offering a practical, tunable approach to robust nonparametric regression and inverse problems. Overall, the framework provides a principled way to hedge against misspecification of priors, noise, and forward operators in high- or infinite-dimensional Bayesian settings.

Abstract

We study a distributionally robust optimization formulation (i.e., a min-max game) for two representative problems in Bayesian nonparametric estimation: Gaussian process regression and, more generally, linear inverse problems. Our formulation seeks the best mean-squared error predictor, in an infinite-dimensional space, against an adversary who chooses the worst-case model in a Wasserstein ball around a nominal infinite-dimensional Bayesian model. The transport cost is chosen to control features such as the degree of roughness of the sample paths that the adversary is allowed to inject. We show that the game has a well-defined value (i.e., strong duality holds in the sense that max-min equals min-max) and that there exists a unique Nash equilibrium which can be computed by a sequence of finite-dimensional approximations. Crucially, the worst-case distribution is itself Gaussian. We explore properties of the Nash equilibrium and the effects of hyperparameters through a set of numerical experiments, demonstrating the versatility of our modeling framework.

Figures (11)

• Figure 1: Correlation functions on $[0,1]^2$ with 10 design points equispaced on $(0,1)$.
• Figure 2: Correlation functions on $[0,1]^2$ with 10 design points equispaced on $(0,0.5)$.
• Figure 3: $95\%$ intervals of sample paths with 10 design points equispaced on $(0,1)$.
• Figure 4: $95\%$ intervals of sample paths with 10 design points equispaced on $(0,0.5)$.
• Figure 5: Sample paths with $\alpha=0.51$ and 10 designs equispaced in $(0,1)$.

Theorems & Definitions (29)

• Example 2.1: Matérn prior ref:lindgren2011explicit
• Example 2.2: RKHS space
• Definition 2.1: Optimal transport cost
• Remark 2.1
• Theorem 3.1: Strong duality for the inverse problem
• Proposition 3.1: Approximation of objective values
• Proposition 3.2: Compactness of the ambiguity set
• Theorem 3.2: Nash equilibrium
• Theorem 3.3: Nash equilibrium property
• Proposition 3.3