Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PINN-BO: A Black-box Optimization Algorithm using Physics-Informed Neural Networks

Dat Phan-Trong, Hung The Tran, Alistair Shilton, Sunil Gupta

TL;DR

The work addresses global black-box optimization where the objective is governed by partial differential equations (PDEs) and evaluations are expensive and noisy. It introduces PINN-BO, a physics-informed neural network-based BO algorithm that incorporates PDE data through a joint loss and uses a PDE-aware exploration scale to drive a Thompson-style acquisition. The authors establish theoretical guarantees via an NTK-PINN-based regret bound that tightens existing results by accounting for the interaction information between function and PDE data. Empirically, PINN-BO outperforms strong GP- and NN-based baselines on synthetic benchmarks and PDE-constrained real-world tasks, demonstrating improved sample efficiency and practical impact in physics-informed optimization.

Abstract

Black-box optimization is a powerful approach for discovering global optima in noisy and expensive black-box functions, a problem widely encountered in real-world scenarios. Recently, there has been a growing interest in leveraging domain knowledge to enhance the efficacy of machine learning methods. Partial Differential Equations (PDEs) often provide an effective means for elucidating the fundamental principles governing the black-box functions. In this paper, we propose PINN-BO, a black-box optimization algorithm employing Physics-Informed Neural Networks that integrates the knowledge from Partial Differential Equations (PDEs) to improve the sample efficiency of the optimization. We analyze the theoretical behavior of our algorithm in terms of regret bound using advances in NTK theory and prove that the use of the PDE alongside the black-box function evaluations, PINN-BO leads to a tighter regret bound. We perform several experiments on a variety of optimization tasks and show that our algorithm is more sample-efficient compared to existing methods.

PINN-BO: A Black-box Optimization Algorithm using Physics-Informed Neural Networks

TL;DR

The work addresses global black-box optimization where the objective is governed by partial differential equations (PDEs) and evaluations are expensive and noisy. It introduces PINN-BO, a physics-informed neural network-based BO algorithm that incorporates PDE data through a joint loss and uses a PDE-aware exploration scale to drive a Thompson-style acquisition. The authors establish theoretical guarantees via an NTK-PINN-based regret bound that tightens existing results by accounting for the interaction information between function and PDE data. Empirically, PINN-BO outperforms strong GP- and NN-based baselines on synthetic benchmarks and PDE-constrained real-world tasks, demonstrating improved sample efficiency and practical impact in physics-informed optimization.

Abstract

Black-box optimization is a powerful approach for discovering global optima in noisy and expensive black-box functions, a problem widely encountered in real-world scenarios. Recently, there has been a growing interest in leveraging domain knowledge to enhance the efficacy of machine learning methods. Partial Differential Equations (PDEs) often provide an effective means for elucidating the fundamental principles governing the black-box functions. In this paper, we propose PINN-BO, a black-box optimization algorithm employing Physics-Informed Neural Networks that integrates the knowledge from Partial Differential Equations (PDEs) to improve the sample efficiency of the optimization. We analyze the theoretical behavior of our algorithm in terms of regret bound using advances in NTK theory and prove that the use of the PDE alongside the black-box function evaluations, PINN-BO leads to a tighter regret bound. We perform several experiments on a variety of optimization tasks and show that our algorithm is more sample-efficient compared to existing methods.
Paper Structure (38 sections, 18 theorems, 85 equations, 7 figures, 1 algorithm)

This paper contains 38 sections, 18 theorems, 85 equations, 7 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Proposed Method
  4. Theoretical Analysis
  5. Proof sketch for Lemma \ref{['lemma:confidence_bound']}
  6. Experimental Results
  7. Baselines
  8. Synthetic Benchmark Functions
  9. Real-world Applications
  10. Optimizing Steady-State Temperature
  11. Optimizing Beam Displacement
  12. Conclusion
  13. Additional Experimental Results
  14. Synthetic Benchmark Functions
  15. Drop-Wave:
  16. ...and 23 more sections

Key Result

Lemma 4.3

Conditioned on $\mathcal{D}_t = \{\mathbf{x}_i, y_i\}_{i=1}^t, \mathcal{R} = \{\mathbf{z}_j, u_j\}_{j=1}^{N_r}$, the acquisition function $\widetilde{f}_t(\mathbf{x}) = h(\mathbf{x}; \boldsymbol{\theta}_{t-1})$ can be viewed as a random draw from a $\mathrm{GP}\left(\mu_{t}^f (\mathbf{x}), \nu_t^2 \ where

Figures (7)

  • Figure 1: The optimization results for synthetic functions comparing the proposed PINN-BO with the baselines. The standard errors are shown by color shading.
  • Figure 2: The optimization results for finding the maximum temperature comparing the proposed PINN-BO with the baselines. It can be seen that, using the PDE heat equation, PINN-BO found the maximum temperature faster than all baselines.
  • Figure 3: The minimum displacement on the non-uniform Euler beam under given loads $q(x)$, flexural rigidity $EI(x)$, and boundary conditions. In comparison with other baselines, our proposed PINN-BO found the location with the smallest displacement, ensuring stability when placing the load over the beam.
  • Figure 4: The optimization results for synthetic functions comparing the proposed PINN-BO with the baselines. The standard errors are shown by color shading.
  • Figure 5: The figures depict the solutions for temperature distributions governed by the heat equation, with each figure corresponding to a specific tuple of boundary conditions described in Section \ref{['section:experiments_2d_laplace']}. It is evident that the region with the highest temperature is relatively small in comparison to the entire domain.
  • ...and 2 more figures

Theorems & Definitions (31)

  • Definition 4.1
  • Lemma 4.3
  • Definition 4.4
  • Lemma 4.4
  • Remark 4.5
  • Lemma 4.5
  • Remark 4.6
  • Remark 4.7
  • Theorem 4.8
  • Lemma B.0
  • ...and 21 more