DiffHybrid-UQ: Uncertainty Quantification for Differentiable Hybrid Neural Modeling

TL;DR

This work tackles uncertainty quantification for differentiable hybrid neural models that fuse known physics with neural surrogates. It introduces DiffHybrid-UQ, combining ensemble-based SWAG for scalable epistemic UQ with the unscented transform for propagating aleatoric uncertainty through nonlinear physics-inspired components within a Bayesian model averaging framework. The approach is demonstrated on both ODEs and PDEs, showing accurate mean predictions and calibrated uncertainty that grows appropriately in data-sparse or extrapolative regions, and enabling parameter inference under aliasing and discretization challenges. The framework supports efficient, parallelizable training and inference, including coarse-grid corrections that preserve uncertainty quantification, offering robust, data-informed predictions for complex physical systems with partial knowledge of the governing laws.

Abstract

The hybrid neural differentiable models mark a significant advancement in the field of scientific machine learning. These models, integrating numerical representations of known physics into deep neural networks, offer enhanced predictive capabilities and show great potential for data-driven modeling of complex physical systems. However, a critical and yet unaddressed challenge lies in the quantification of inherent uncertainties stemming from multiple sources. Addressing this gap, we introduce a novel method, DiffHybrid-UQ, for effective and efficient uncertainty propagation and estimation in hybrid neural differentiable models, leveraging the strengths of deep ensemble Bayesian learning and nonlinear transformations. Specifically, our approach effectively discerns and quantifies both aleatoric uncertainties, arising from data noise, and epistemic uncertainties, resulting from model-form discrepancies and data sparsity. This is achieved within a Bayesian model averaging framework, where aleatoric uncertainties are modeled through hybrid neural models. The unscented transformation plays a pivotal role in enabling the flow of these uncertainties through the nonlinear functions within the hybrid model. In contrast, epistemic uncertainties are estimated using an ensemble of stochastic gradient descent (SGD) trajectories. This approach offers a practical approximation to the posterior distribution of both the network parameters and the physical parameters. Notably, the DiffHybrid-UQ framework is designed for simplicity in implementation and high scalability, making it suitable for parallel computing environments. The merits of the proposed method have been demonstrated through problems governed by both ordinary and partial differentiable equations.

Figures (20)

• Figure 1: The overview of the auto-regressive DiffHybrid architecture for uncertainty propagation.
• Figure 2: Illustrative plot of unscented transformation (UT) versus Monte Carlo (MC) methods for uncertainty propagation.
• Figure 3: (a) A schematic of the estimation of the posterior distribution using the ensemble-based SWAG training (b) Prediction/Inference using the posterior distribution
• Figure 4: Compares the DiffHybrid-UQ model's prediction and UQ against HMC method. Training data spans $\mathcal{D} = (x_1, x_2 \in \mathbb{R}^1 \times [0:0.1:20 s])$ with testing in the $(x_1, x_2 \in \mathbb{R}^1 \times (20:0.1:30 s])$ region.
• Figure 5: DiffHybrid-UQ prediction with quantified uncertainties compared against the ground truth for different testing initial conditions. Trained initial condition: $\mathbf{x}=[0,15]$.