Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bayesian Interpolating Neural Network (B-INN): a scalable and reliable Bayesian model for large-scale physical systems

Chanwook Park, Brian Kim, Jiachen Guo, Wing Kam Liu

TL;DR

This paper tackles the need for scalable, uncertainty-aware surrogates in large-scale physical systems by introducing the Bayesian Interpolating Neural Network (B-INN), which fuses INN interpolation, tensor decomposition, and Bayesian linear regression. It proves that B-INN is contained within the GP function space, and that inference scales linearly with the training size, enabling ultra-fast uncertainty quantification. Through synthetic 1D tests, the BlendedNet aerodynamics dataset, and high-dimensional active-learning experiments on Poisson and heat equations, B-INN achieves 20–10,000x speedups over Bayesian neural networks and Gaussian processes while maintaining or improving predictive accuracy and well-calibrated uncertainties. These results demonstrate B-INN’s practicality for uncertainty-driven active learning in industry-scale simulations, with potential for rapid deployment in design optimization and digital twins.

Abstract

Neural networks and machine learning models for uncertainty quantification suffer from limited scalability and poor reliability compared to their deterministic counterparts. In industry-scale active learning settings, where generating a single high-fidelity simulation may require days or weeks of computation and produce data volumes on the order of gigabytes, they quickly become impractical. This paper proposes a scalable and reliable Bayesian surrogate model, termed the Bayesian Interpolating Neural Network (B-INN). The B-INN combines high-order interpolation theory with tensor decomposition and alternating direction algorithm to enable effective dimensionality reduction without compromising predictive accuracy. We theoretically show that the function space of a B-INN is a subset of that of Gaussian processes, while its Bayesian inference exhibits linear complexity, $\mathcal{O}(N)$, with respect to the number of training samples. Numerical experiments demonstrate that B-INNs can be from 20 times to 10,000 times faster with a robust uncertainty estimation compared to Bayesian neural networks and Gaussian processes. These capabilities make B-INN a practical foundation for uncertainty-driven active learning in large-scale industrial simulations, where computational efficiency and robust uncertainty calibration are paramount.

Bayesian Interpolating Neural Network (B-INN): a scalable and reliable Bayesian model for large-scale physical systems

TL;DR

This paper tackles the need for scalable, uncertainty-aware surrogates in large-scale physical systems by introducing the Bayesian Interpolating Neural Network (B-INN), which fuses INN interpolation, tensor decomposition, and Bayesian linear regression. It proves that B-INN is contained within the GP function space, and that inference scales linearly with the training size, enabling ultra-fast uncertainty quantification. Through synthetic 1D tests, the BlendedNet aerodynamics dataset, and high-dimensional active-learning experiments on Poisson and heat equations, B-INN achieves 20–10,000x speedups over Bayesian neural networks and Gaussian processes while maintaining or improving predictive accuracy and well-calibrated uncertainties. These results demonstrate B-INN’s practicality for uncertainty-driven active learning in industry-scale simulations, with potential for rapid deployment in design optimization and digital twins.

Abstract

Neural networks and machine learning models for uncertainty quantification suffer from limited scalability and poor reliability compared to their deterministic counterparts. In industry-scale active learning settings, where generating a single high-fidelity simulation may require days or weeks of computation and produce data volumes on the order of gigabytes, they quickly become impractical. This paper proposes a scalable and reliable Bayesian surrogate model, termed the Bayesian Interpolating Neural Network (B-INN). The B-INN combines high-order interpolation theory with tensor decomposition and alternating direction algorithm to enable effective dimensionality reduction without compromising predictive accuracy. We theoretically show that the function space of a B-INN is a subset of that of Gaussian processes, while its Bayesian inference exhibits linear complexity, , with respect to the number of training samples. Numerical experiments demonstrate that B-INNs can be from 20 times to 10,000 times faster with a robust uncertainty estimation compared to Bayesian neural networks and Gaussian processes. These capabilities make B-INN a practical foundation for uncertainty-driven active learning in large-scale industrial simulations, where computational efficiency and robust uncertainty calibration are paramount.
Paper Structure (33 sections, 2 theorems, 75 equations, 6 figures, 4 tables, 1 algorithm)

This paper contains 33 sections, 2 theorems, 75 equations, 6 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Scalable Gaussian processes (GPs)
  4. Bayesian neural networks (BNNs)
  5. Diffusion models for uncertainty quantification
  6. Bayesian Interpolating Neural Network (B-INN)
  7. Review of INN
  8. Bayesian linear regression
  9. Bayesian inference of INNs
  10. Equivalence between GP and B-INN
  11. Predictive mean and variance of B-INN
  12. Results
  13. Regression with uncertainty quantification
  14. GP, BNN, and B-INN in 1D input space
  15. Learning open source aerodynamics dataset - BlendedNet
  16. ...and 18 more sections

Key Result

Theorem 3.2

Assume Assumption assump:weights_main. Let $y^{(M)}$ denote the B-INN output $y$ in Eq. eq:INN_ND when the number of modes is $M$, and define the normalization process Then, as $M\to\infty$, $\bar{y}^{(M)}$ converges in finite-dimensional distributions to a centered Gaussian process with kernel where $\phi_{d.j}$ is the standard RBF basis function.

Figures (6)

  • Figure 1: INN architecture for a one-input and one-output system where the 1D input space is discretized with $J=6$ grid points. Gaussian functions are used for $\phi_{j}(x)$park2025interpolating.
  • Figure 2: INN architecture for a three-input and one-output system park2025interpolating
  • Figure 3: Regression results of GP, B-INNs, and BNNs. Model hyperparameters can be found in Appendix \ref{['appendix:model_hyperparameters']}.
  • Figure 4: Training time (a) and test RMSE (b) as the number of training samples increases.
  • Figure 5: Test data (a), B-INN prediction (b), and normalized absolute error (c) of force coefficient $(C_{f_z})$ of BlendedNet dataset.
  • ...and 1 more figures

Theorems & Definitions (3)

  • Theorem 3.2: As $M\to\infty$, the normalized B-INN prior converges to a GP prior in finite-dimensional distributions
  • Theorem 3.3: As $M\to\infty$, the normalized B-INN posterior at finitely many test points converges weakly to the GP regression posterior
  • proof