Bi-fidelity Variational Auto-encoder for Uncertainty Quantification

TL;DR

The paper tackles uncertainty quantification for quantities of interest when high-fidelity simulations are expensive. It proposes BF-VAE, a variational auto-encoder that leverages abundant low-fidelity data through a latent-space auto-regressive link to a high-fidelity latent variable, enabling HF QoI generation with limited HF samples. The framework introduces the BF-ELBO and an information-theoretic BF-IB interpretation, and it demonstrates improved HF pdf accuracy across three PDE problems (composite beam, cavity flow, Burgers' equation) compared to HF-only VAEs. This approach reduces HF data requirements while maintaining accuracy, and its latent-space bi-fidelity coupling broadens applicability to high-dimensional QoIs and other generative models for UQ.

Abstract

Quantifying the uncertainty of quantities of interest (QoIs) from physical systems is a primary objective in model validation. However, achieving this goal entails balancing the need for computational efficiency with the requirement for numerical accuracy. To address this trade-off, we propose a novel bi-fidelity formulation of variational auto-encoders (BF-VAE) designed to estimate the uncertainty associated with a QoI from low-fidelity (LF) and high-fidelity (HF) samples of the QoI. This model allows for the approximation of the statistics of the HF QoI by leveraging information derived from its LF counterpart. Specifically, we design a bi-fidelity auto-regressive model in the latent space that is integrated within the VAE's probabilistic encoder-decoder structure. An effective algorithm is proposed to maximize the variational lower bound of the HF log-likelihood in the presence of limited HF data, resulting in the synthesis of HF realizations with a reduced computational cost. Additionally, we introduce the concept of the bi-fidelity information bottleneck (BF-IB) to provide an information-theoretic interpretation of the proposed BF-VAE model. Our numerical results demonstrate that BF-VAE leads to considerably improved accuracy, as compared to a VAE trained using only HF data, when limited HF data is available.

Figures (16)

• Figure 1: Instead of conducting bi-fidelity regression directly in high-dimensional observation space (blue path), we introduce an approach via low-dimensional latent space (red path).
• Figure 2: The probabilistic encoder $q_{\bm\phi}(\bm z\vert\bm x)$ of a VAE produces two separate vectors, $\bm\mu_{\bm\phi}(\bm x)$ and $\bm\sigma_{\bm\phi}(\bm x)$, which respectively represent the mean and standard deviation of resulting latent variable $\bm z$ following a multivariate Gaussian distribution. The random vector $\bm\varepsilon\sim\mathcal{N}(\bm 0,\bm I)$ provides randomness for the encoder output $\bm z$ and is used for the reparameterization trick in Equation \ref{['eq:repara2']}.
• Figure 3: Structure of the proposed BF-VAE model. The probabilistic encoder $q_{\bm\phi}(\bm z^L\vert\bm x^L)$ produces two independent vectors, $\mu_{\bm\phi}(\bm x^L)$ and $\sigma_{\bm\phi}(\bm x^L)$, which represent the mean and standard deviation of a resulting multivariate Gaussian. The latent auto-regression $p_{\bm\psi}(\bm z^H\vert\bm z^L)$ is a simplified single-layer neural network $\bm K_{\bm\psi}$ defined in Equation \ref{['eq:latent-layer-K']} added with a noise $\gamma\bm\eta$. The probabilistic decoder $p_{\bm\theta}(\bm x^H\vert\bm z^H)$ is pre-trained by LF data via the transfer learning technique, with its last layer tuned by LF and HF data pairs. White circles are random vectors and colored blocks are parameterized components for training. Blue blocks are solely trained by LF data and green blocks are trained by both LF and HF data.
• Figure 4: The bi-fidelity information bottleneck architecture has an encoder and a decoder, impacted by the information compression function $\mathbb{I}(\bm x^L,\bm z_{\bm\psi})$ and information preservation function $\mathbb{I}(\bm z_{\bm\psi},\bm x^H)$, respectively. The random vector $\bm z_{\bm\psi}$ is designed to disclose the relation between LF and HF data in the latent space. The bottleneck part is necessary since only a limited number of HF realizations are available for learning the relationship between LF and HF data.
• Figure 5: Cantilever beam (left) and the composite cross section (right) adapted from hampton2018practical.

Theorems & Definitions (5)

• Definition 3.1
• proof
• proof
• Proposition C.1
• Proposition C.2