Deep Jump Gaussian Processes for Surrogate Modeling of High-Dimensional Piecewise Continuous Functions

TL;DR

DJGP addresses the challenge of modeling high-dimensional piecewise continuous functions by embedding a locally linear projection within Jump Gaussian Processes and tying these projections with a Gaussian Process prior across input space. The method uses a two-layer GP/JGP architecture and a carefully constructed variational inference scheme with local and global inducing variables to enable scalable training and accurate uncertainty quantification. Empirical results on synthetic and real-world datasets show DJGP consistently improves predictive accuracy and uncertainty calibration (RMSE and CRPS) over strong baselines, including JGP with various dimension-reduction strategies and deep GPs. The work demonstrates a practical approach to nonstationary, discontinuous surrogates in higher dimensions and highlights potential for online extensions and richer boundary modeling in future work.

Abstract

We introduce Deep Jump Gaussian Processes (DJGP), a novel method for surrogate modeling of high-dimensional piecewise continuous functions. DJGP overcomes the limitations of conventional Jump Gaussian Processes in high-dimensional input spaces by adding a locally linear projection layer to Jump Gaussian Processes. This projection uses region-specific matrices to capture local subspace structures, naturally complementing the localized nature of JGP, a variant of local Gaussian Processes. To control model complexity, we place a Gaussian Process prior on the projection matrices, allowing them to evolve smoothly across the input space. The projected inputs are then modeled with a JGP to capture piecewise continuous relationships with the response. This yields a distinctive two-layer deep learning of GP/JGP. We further develop a scalable variational inference algorithm to jointly learn the projection matrices and JGP hyperparameters. Experiments on synthetic and benchmark datasets demonstrate that DJGP delivers superior predictive accuracy and more reliable uncertainty quantification compared to existing approaches.

Figures (8)

• Figure 1: Illustrations of response surfaces over 2D latent spaces. (a) A noiseless ground-truth surface with sharp transitions from the L2 Dataset. (b) A simulated surface from the high-dimensional LH dataset projected onto 2D.
• Figure 2: Effect of observed dimension $D$ on RMSE under fixed latent dimension $K=5$ and varying training sizes. DJGP maintains stable performance even with increased dimensionality.
• Figure 3: Effect of training set size $N$ on RMSE under fixed observed dimension $D=30$ and different latent dimensions. Larger $N$ leads to denser local neighborhoods and improved performance.
• Figure 4: Effect of latent dimension $K$ on DJGP performance. (Left) Mean RMSE as a function of the latent dimension $K$ under varying sample sizes $N$. (Right) RMSE exhibits a scaling trend governed by $N^{K}$ rather than $N^{D}$, consistent with the theoretical error behavior of stationary Gaussian processes with intrinsic input dimension $K$park2022jump. This indicates that DJGP effectively identifies the latent subspace and mitigates the curse of dimensionality in high-dimensional observations.
• Figure 5: Effect of local and global inducing point counts $(L_1, L_2)$ on (a) RMSE and (b) CRPS. The configuration $(4, 40)$ achieves the best balance between predictive accuracy and uncertainty estimation.