Gradient-based Active Learning with Gaussian Processes for Global Sensitivity Analysis

TL;DR

This work builds on recent advances in active learning for sensitivity analysis by developing acquisition functions that better account for correlations between partial derivatives and their impact on the response surface, leading to a more comprehensive and robust methodology than existing DGSM-oriented criteria.

Abstract

Global sensitivity analysis of complex numerical simulators is often limited by the small number of model evaluations that can be afforded. In such settings, surrogate models built from a limited set of simulations can substantially reduce the computational burden, provided that the design of computer experiments is enriched efficiently. In this context, we propose an active learning approach that, for a fixed evaluation budget, targets the most informative regions of the input space to improve sensitivity analysis accuracy. More specifically, our method builds on recent advances in active learning for sensitivity analysis (Sobol' indices and derivative-based global sensitivity measures, DGSM) that exploit derivatives obtained from a Gaussian process (GP) surrogate. By leveraging the joint posterior distribution of the GP gradient, we develop acquisition functions that better account for correlations between partial derivatives and their impact on the response surface, leading to a more comprehensive and robust methodology than existing DGSM-oriented criteria. The proposed approach is first compared to state-of-the-art methods on standard benchmark functions, and is then applied to a real environmental model of pesticide transfers.

Figures (14)

• Figure 1: Aggregated MSE of first-order $S_1$ and total-order Sobol' indices $S_T$ for the $d=10$ G-Sobol function, estimated from Gaussian process surrogates trained on different designs of computer experiments. Each point corresponds to a distinct training design, and colors indicate the size of the design (from $5d$ up to $50d$). For each surrogate, Sobol' indices are computed using the same Monte Carlo sample, so differences in MSE solely reflect the impact of the training design. As expected, larger designs generally yield more accurate Sobol' estimates, although some small designs (e.g., of size $5d$) occasionally produce surrogates with comparable accuracy.
• Figure 2: Gaussian process regression on a 1D toy function. (Top) Posterior distribution of $\eta$. (Bottom) Posterior distribution of $\nabla\eta$.
• Figure 3: Gaussian process posterior and look-ahead visualization. Current posterior (grey) and a set of fantasy posteriors obtained by conditioning on hypothetical evaluations at the candidate locations $X_{\mathrm{cand}}$. All fantasy trajectories are shown as thin lines, while one randomly selected fantasy path is highlighted in bold and the corresponding fantasy observation at $X_{\mathrm{cand}}$ is indicated by the marker.
• Figure 4: Heatmaps of the joint posterior covariance of the GP gradient at the sampled locations $\mathbf{X}_s$: the exact covariance $\Sigma_\nabla$, its block-diagonal approximation $\widetilde{\Sigma}_\nabla$ obtained by retaining only within-cluster blocks (classical KMeans on $\mathbf X_s$), and the residual $E=\Sigma_\nabla-\widetilde{\Sigma}_\nabla$. The horizontal and vertical axes index the same flattened gradient vector $Z_{\mathbf X_s}=[\nabla\eta(x_1^*)^\top,\ldots,\nabla\eta(x_N^*)^\top]^\top\in\mathbb R^{Nd}$, so each pixel corresponds to the covariance between two partial derivatives (point and coordinate). The color scale is a symmetric logarithmic normalization.
• Figure 5: Active learning strategies on a selection of classical test functions.

Theorems & Definitions (10)

• Remark 2.1
• Proposition 4.1
• Lemma B.1
• proof
• Proposition B.1
• proof
• Lemma B.2: Monotonicity of $h$
• proof
• Lemma B.3: Spectral norm
• proof