BITS for GAPS: Bayesian Information-Theoretic Sampling for hierarchical GAussian Process Surrogates

TL;DR

BITS for GAPS addresses data-sparse hybrid modeling by placing a Bayesian hierarchical Gaussian Process prior over latent residuals and employing entropy-based acquisition to drive informative data collection. The authors derive a tractable, closed-form-like entropy approximation for the predictive posterior, enabling efficient selection of input designs via a Gaussian Mixture Model representation. The framework is demonstrated on a VLE/ distillation design problem, where the GP surrogate for activity coefficients is embedded into extended Raoult’s law to produce phase envelopes and guide column design. Results show improved sample efficiency, reduced predictive bias, and robust uncertainty quantification, with MCMC diagnostics confirming well-posed inference. Overall, BITS for GAPS offers a principled, uncertainty-aware approach that blends physical structure with probabilistic surrogates for complex, non-ideal systems.

Abstract

We introduce the Bayesian Information-Theoretic Sampling for hierarchical GAussian Process Surrogates (BITS for GAPS) framework to emulate latent components in hybrid physical systems. BITS for GAPS supports serial hybrid modeling, where known physics governs part of the system and residual dynamics are represented as a latent function inferred from data. A Gaussian process prior is placed over the latent function, with hierarchical priors on its hyperparameters to encode physically meaningful structure in the predictive posterior. To guide data acquisition, we derive entropy-based acquisition functions that quantify expected information gain from candidate input locations, identifying samples most informative for training the surrogate. Specifically, we obtain a closed-form expression for the differential entropy of the predictive posterior and establish a tractable lower bound for efficient evaluation. These derivations approximate the predictive posterior as a finite, uniformly weighted mixture of Gaussian processes. We demonstrate the framework's utility by modeling activity coefficients in vapor-liquid equilibrium systems, embedding the surrogate into extended Raoult's law for distillation design. Numerical results show that entropy-guided sampling improves sample efficiency by targeting regions of high uncertainty and potential information gain. This accelerates surrogate convergence, enhances predictive accuracy in non-ideal regimes, and preserves physical consistency. Overall, BITS for GAPS provides an efficient, interpretable, and uncertainty-aware framework for hybrid modeling of complex physical systems.

Figures (12)

• Figure 1: Overview of the BITS for GAPS framework.
• Figure 2: Training and testing data for the activity coefficient surrogate model. (a) Latin hypercube sample of $\text{PrOH}$ mole fraction, $z\xspace_{\text{PrOH}\xspace}$ [ ], and temperature, $T$ [K]. Red circles (blue triangles) indicate selected design points for training (testing). (b) Activity coefficient, $\gamma$ [ ], as a function of $z\xspace_{\text{PrOH}\xspace}$ and $T$, at atmospheric pressure. $\text{PrOH}$ is shown in orange; $\text{H}_2\text{O}$ in green.
• Figure 3: Posterior differential entropy, $\mathcal{H}\xspace\{f\xspace(\mathbf{x\xspace}\xspace_*)\mid \mathbf{y\xspace}\xspace\}$, as a function of temperature, $T$ [K], and mole fraction of $\text{PrOH}$, $z\xspace_{\text{PrOH}\xspace}~[~\,]$, at iterations 1-6 (a-f). White circles denote previously sampled (training) points, and red squares denote the locations selected by the optimizer as having maximum posterior entropy. The red squares are sampled and augmented into the training data for subsequent iterations.
• Figure 4: Maximum entropy and minimum information over successive iterations of BITS for GAPS. The black solid line shows the maximum entropy across evaluated candidate models at each iteration. The blue dashed line shows the corresponding minimum information (defined as the negative of maximum entropy).
• Figure 5: Surrogate model performance at early and late stages of BITS for GAPS. Left column (a, c): Results from iteration 1. Right column (b, d): Results from iteration 10. (a–b) Parity plots comparing surrogate model predictions and ground truth values from the Wilson (ground truth) model for training (red circles) and test (blue downward triangles) datasets. Error bars represent a 90% credible interval (CI). The dashed black line indicates the ideal parity (1:1) line. Insets provide a zoomed-in view of the lower range, with zoom regions highlighted by dashed rectangles. (c–d) Box plots showing the distribution of mean absolute error (MAE) and root mean square error (RMSE) for training (red) and test (blue) datasets.