Empowering Bayesian Neural Networks with Functional Priors through Anchored Ensembling for Mechanics Surrogate Modeling Applications

TL;DR

The paper tackles uncertainty quantification in mechanics surrogate modeling by embedding function-space priors into Bayesian neural networks through anchored ensembles with correlated regularization. It shows that parameter-space priors derived from functional priors are typically anisotropic and low-rank, with weight correlations being crucial for faithful prior transfer, and proposes a MAP training scheme using a learned multivariate Gaussian prior $p_0(\boldsymbol{\omega}) = \mathcal{N}(\boldsymbol{\omega}; \boldsymbol{\mu}_0, \Sigma_0)$ whose covariance is obtained from a low-rank SVD of pre-training data. A novel algorithm leveraging this structure demonstrates improved mean predictions and well-calibrated epistemic uncertainty on both a 1D interpolation/extrapolation task and a multi-output materials surrogate, including robust performance under out-of-distribution conditions. These findings offer practical guidance for designing functional priors in mechanics and enable scalable, principled uncertainty quantification in physics-informed surrogates.

Abstract

In recent years, neural networks (NNs) have become increasingly popular for surrogate modeling tasks in mechanics and materials modeling applications. While traditional NNs are deterministic functions that rely solely on data to learn the input--output mapping, casting NN training within a Bayesian framework allows to quantify uncertainties, in particular epistemic uncertainties that arise from lack of training data, and to integrate a priori knowledge via the Bayesian prior. However, the high dimensionality and non-physicality of the NN parameter space, and the complex relationship between parameters (NN weights) and predicted outputs, renders both prior design and posterior inference challenging. In this work we present a novel BNN training scheme based on anchored ensembling that can integrate a priori information available in the function space, from e.g. low-fidelity models. The anchoring scheme makes use of low-rank correlations between NN parameters, learnt from pre-training to realizations of the functional prior. We also perform a study to demonstrate how correlations between NN weights, which are often neglected in existing BNN implementations, is critical to appropriately transfer knowledge between the function-space and parameter-space priors. Performance of our novel BNN algorithm is first studied on a small 1D example to illustrate the algorithm's behavior in both interpolation and extrapolation settings. Then, a thorough assessment is performed on a multi--input--output materials surrogate modeling example, where we demonstrate the algorithm's capabilities both in terms of accuracy and quality of the uncertainty estimation, for both in-distribution and out-of-distribution data.

Figures (14)

• Figure 1: Functional priors considered for the function- to parameter-space prior study. Each gray curve represents a realization $g^{[k]}$ from the functional prior. The colored lines represent two NNs fitted to two specific realizations.
• Figure 2: Histogram of NN weights (kernels) pre-trained to priors B and D.
• Figure 3: Correlation between some of the NN weights pre-trained to priors B and D.
• Figure 4: Reconstructing functional priors different parameter-space prior distributions: a) zero-mean isotropic Gaussian, b) non-zero mean factorized Gaussian, c) non-zero mean factorized generalized normal, d) multivariate-Gaussian.
• Figure 5: Decay of singular values of $\mathbb{W}_0$, matrix of NN weights pre-trained to different priors.