Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bayesian Entropy Neural Networks for Physics-Aware Prediction

Rahul Rathnakumar, Jiayu Huang, Hao Yan, Yongming Liu

TL;DR

The paper tackles the challenge of embedding physical and informational constraints into deep neural surrogates while preserving meaningful uncertainty quantification. It introduces Bayesian Entropy Neural Networks (BENN), which merge maximum entropy principles with Bayesian neural networks and employ variational inference plus the Modified Differential Method of Multipliers to enforce multiple, potentially conflicting constraints on predictions, derivatives, and predictive variance. The approach is demonstrated on 1D regression, beam deflection, and microstructure generation via a BE-CVAE, showing improved constraint satisfaction and competitive performance relative to constrained baselines. This framework offers a principled, scalable route to physics-aware, uncertainty-aware surrogate modeling with broad applicability in engineering and materials science.

Abstract

This paper addresses the need for deep learning models to integrate well-defined constraints into their outputs, driven by their application in surrogate models, learning with limited data and partial information, and scenarios requiring flexible model behavior to incorporate non-data sample information. We introduce Bayesian Entropy Neural Networks (BENN), a framework grounded in Maximum Entropy (MaxEnt) principles, designed to impose constraints on Bayesian Neural Network (BNN) predictions. BENN is capable of constraining not only the predicted values but also their derivatives and variances, ensuring a more robust and reliable model output. To achieve simultaneous uncertainty quantification and constraint satisfaction, we employ the method of multipliers approach. This allows for the concurrent estimation of neural network parameters and the Lagrangian multipliers associated with the constraints. Our experiments, spanning diverse applications such as beam deflection modeling and microstructure generation, demonstrate the effectiveness of BENN. The results highlight significant improvements over traditional BNNs and showcase competitive performance relative to contemporary constrained deep learning methods.

Bayesian Entropy Neural Networks for Physics-Aware Prediction

TL;DR

The paper tackles the challenge of embedding physical and informational constraints into deep neural surrogates while preserving meaningful uncertainty quantification. It introduces Bayesian Entropy Neural Networks (BENN), which merge maximum entropy principles with Bayesian neural networks and employ variational inference plus the Modified Differential Method of Multipliers to enforce multiple, potentially conflicting constraints on predictions, derivatives, and predictive variance. The approach is demonstrated on 1D regression, beam deflection, and microstructure generation via a BE-CVAE, showing improved constraint satisfaction and competitive performance relative to constrained baselines. This framework offers a principled, scalable route to physics-aware, uncertainty-aware surrogate modeling with broad applicability in engineering and materials science.

Abstract

This paper addresses the need for deep learning models to integrate well-defined constraints into their outputs, driven by their application in surrogate models, learning with limited data and partial information, and scenarios requiring flexible model behavior to incorporate non-data sample information. We introduce Bayesian Entropy Neural Networks (BENN), a framework grounded in Maximum Entropy (MaxEnt) principles, designed to impose constraints on Bayesian Neural Network (BNN) predictions. BENN is capable of constraining not only the predicted values but also their derivatives and variances, ensuring a more robust and reliable model output. To achieve simultaneous uncertainty quantification and constraint satisfaction, we employ the method of multipliers approach. This allows for the concurrent estimation of neural network parameters and the Lagrangian multipliers associated with the constraints. Our experiments, spanning diverse applications such as beam deflection modeling and microstructure generation, demonstrate the effectiveness of BENN. The results highlight significant improvements over traditional BNNs and showcase competitive performance relative to contemporary constrained deep learning methods.
Paper Structure (23 sections, 28 equations, 6 figures, 2 tables)

This paper contains 23 sections, 28 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Overview of Bayesian Neural Networks
  4. Incorporating Constraint Information in Bayesian Entropy Method
  5. Bayesian Entropy Neural Networks
  6. Experiments
  7. 1D-Regression
  8. Incorporating Value and Derivative Constraints to the Prediction
  9. Value Constraints Outside Training Range:
  10. Derivative constraints
  11. Variance constraints
  12. Conflicting Constraints
  13. Bound Constraints
  14. Beam Deflection Problem
  15. Constraint Implementation
  16. ...and 8 more sections

Figures (6)

  • Figure 1: 1-D constrained regression demonstrations with value, derivative and bound constraints
  • Figure 2: 1-D constrained regression demonstrations using PR-BNN with value, derivative and bound constraints
  • Figure 3: 1-D constrained regression demonstrations with variance constraints outside the training data
  • Figure 4: Beam configuration. The observable regions are given in the dashed boxes.
  • Figure 5: Predicted beam deflection with BENN and BNN.
  • ...and 1 more figures