Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gaussian Process Regression with Soft Inequality and Monotonicity Constraints

Didem Kochan, Xiu Yang

TL;DR

This work introduces a probabilistic, soft-constraint framework for Gaussian Process Regression that enforces physical inequalities and monotonicity via a quantum-inspired Hamiltonian Monte Carlo (QHMC) trainer. By treating constraint satisfaction as a probabilistic objective and selecting constraint locations adaptively, the method preserves Gaussian posteriors while improving accuracy and reducing uncertainty. Theoretical analysis confirms convergence of QHMC to the true posterior and that finite constraint sets preserve convergence, while extensive numerical experiments across 2D–20D and 3D PDE-inspired problems demonstrate substantial time savings (about 20%) and accuracy gains (around 15%) over traditional truncated-Gaussian approaches, particularly in high dimensions and noisy settings. The approach is applicable to a wide range of physical systems requiring inequality and monotonicity constraints, with monotonicity enforced through derivative-GP formulations and adaptive active-learning of constraint points.

Abstract

Gaussian process (GP) regression is a non-parametric, Bayesian framework to approximate complex models. Standard GP regression can lead to an unbounded model in which some points can take infeasible values. We introduce a new GP method that enforces the physical constraints in a probabilistic manner. This GP model is trained by the quantum-inspired Hamiltonian Monte Carlo (QHMC). QHMC is an efficient way to sample from a broad class of distributions. Unlike the standard Hamiltonian Monte Carlo algorithm in which a particle has a fixed mass, QHMC allows a particle to have a random mass matrix with a probability distribution. Introducing the QHMC method to the inequality and monotonicity constrained GP regression in the probabilistic sense, our approach improves the accuracy and reduces the variance in the resulting GP model. According to our experiments on several datasets, the proposed approach serves as an efficient method as it accelerates the sampling process while maintaining the accuracy, and it is applicable to high dimensional problems.

Gaussian Process Regression with Soft Inequality and Monotonicity Constraints

TL;DR

This work introduces a probabilistic, soft-constraint framework for Gaussian Process Regression that enforces physical inequalities and monotonicity via a quantum-inspired Hamiltonian Monte Carlo (QHMC) trainer. By treating constraint satisfaction as a probabilistic objective and selecting constraint locations adaptively, the method preserves Gaussian posteriors while improving accuracy and reducing uncertainty. Theoretical analysis confirms convergence of QHMC to the true posterior and that finite constraint sets preserve convergence, while extensive numerical experiments across 2D–20D and 3D PDE-inspired problems demonstrate substantial time savings (about 20%) and accuracy gains (around 15%) over traditional truncated-Gaussian approaches, particularly in high dimensions and noisy settings. The approach is applicable to a wide range of physical systems requiring inequality and monotonicity constraints, with monotonicity enforced through derivative-GP formulations and adaptive active-learning of constraint points.

Abstract

Gaussian process (GP) regression is a non-parametric, Bayesian framework to approximate complex models. Standard GP regression can lead to an unbounded model in which some points can take infeasible values. We introduce a new GP method that enforces the physical constraints in a probabilistic manner. This GP model is trained by the quantum-inspired Hamiltonian Monte Carlo (QHMC). QHMC is an efficient way to sample from a broad class of distributions. Unlike the standard Hamiltonian Monte Carlo algorithm in which a particle has a fixed mass, QHMC allows a particle to have a random mass matrix with a probability distribution. Introducing the QHMC method to the inequality and monotonicity constrained GP regression in the probabilistic sense, our approach improves the accuracy and reduces the variance in the resulting GP model. According to our experiments on several datasets, the proposed approach serves as an efficient method as it accelerates the sampling process while maintaining the accuracy, and it is applicable to high dimensional problems.
Paper Structure (19 sections, 1 theorem, 44 equations, 17 figures, 6 tables, 1 algorithm)

This paper contains 19 sections, 1 theorem, 44 equations, 17 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Gaussian Process under inequality constraints
  3. Standard GP regression framework
  4. Quantum-inspired Hamiltonian Monte Carlo
  5. Proposed method
  6. Enforcing Monotonicity Constraints
  7. Theoretical analysis of the method
  8. Convergence of QHMC training
  9. Convergence properties of probabilistic approach
  10. Numerical examples
  11. Inequaltiy Constraints
  12. Example 1
  13. Example 2
  14. Example 3: Solute transport in heterogeneous porous media
  15. Example 4: Heat Transfer in a Hallow Sphere
  16. ...and 4 more sections

Key Result

Theorem 1

$v_{m}$ converges to $v$ as $\mathcal{P}_{m}$ converges to $\mathcal{P}$, that is

Figures (17)

  • Figure 1: Relative error of the algorithms with different data sizes and signal to noise ratios (SNR) for Example 1 (2D), inequality.
  • Figure 2: Execution times (in seconds) of the algorithms with different signal to noise ratios (SNR) and datasizes for Example 1 (2D), inequality.
  • Figure 3: Posterior variances of the algorithms with different signal to noise ratios (SNR) and datasizes for Example 1 (2D), inequality.
  • Figure 4: Relative error of the algorithms with different data sizes and signal to noise ratios (SNR) for Example 2 (10D), inequality.
  • Figure 5: Execution times (in minutes) of the algorithms with different signal to noise ratios (SNR) and datasizes for Example 2 (10D), inequality.
  • ...and 12 more figures

Theorems & Definitions (2)

  • Theorem 1
  • proof