Uncertainty Quantification of Graph Convolution Neural Network Models of Evolving Processes

TL;DR

Uncertainty quantification for neural surrogates in evolving spatial-temporal processes is challenging due to high dimensionality and nonconvex posteriors. The authors compare three Bayesian UQ methods—Hamiltonian Monte Carlo, Stein variational gradient descent, and projected SVGD—applied to GCNN-based models of polycrystal mechanics and gas diffusion. They show that SVGD reproduces posterior-predictive uncertainty profiles similar to HMC, while requiring fewer resources, whereas pSVGD offers subspace-based efficiency with stability trade-offs in high-dimensional settings. The study provides detailed analyses of posterior structure, covariance patterns, and predictive fidelity across data regimes, supporting SVGD as a viable scalable UQ approach for SciML evolving-process models.

Abstract

The application of neural network models to scientific machine learning tasks has proliferated in recent years. In particular, neural network models have proved to be adept at modeling processes with spatial-temporal complexity. Nevertheless, these highly parameterized models have garnered skepticism in their ability to produce outputs with quantified error bounds over the regimes of interest. Hence there is a need to find uncertainty quantification methods that are suitable for neural networks. In this work we present comparisons of the parametric uncertainty quantification of neural networks modeling complex spatial-temporal processes with Hamiltonian Monte Carlo and Stein variational gradient descent and its projected variant. Specifically we apply these methods to graph convolutional neural network models of evolving systems modeled with recurrent neural network and neural ordinary differential equations architectures. We show that Stein variational inference is a viable alternative to Monte Carlo methods with some clear advantages for complex neural network models. For our exemplars, Stein variational interference gave similar uncertainty profiles through time compared to Hamiltonian Monte Carlo, albeit with generally more generous variance.Projected Stein variational gradient descent also produced similar uncertainty profiles to the non-projected counterpart, but large reductions in the active weight space were confounded by the stability of the neural network predictions and the convoluted likelihood landscape.

Figures (20)

• Figure 1: Illustration of likelihood and samples from HMC, SVGD, and pSVGD for MVN (upper row) and Tor (lower row). The base shows samples projected to the $w_1$-$w_2$ plane. Left to right: HMC, SVGD, pSVGD, diffusion map eigenvectors.
• Figure 2: Crystal plasticity. Upper panel: realizations of the orientation field $\phi$. Lower panel: corresponding $\mathbf{S}_{11}$ field at maximum applied tensile ($\boldsymbol{\epsilon}_{11}$) strain.
• Figure 3: Crystal plasticity: normalized stress response curves $\bar{\mathbf{S}}(t)$ for different configurational realizations (${\boldsymbol{\phi}}(\mathbf{X})$).
• Figure 4: GCNN-GRU model of the crystal plasticity data, $N_w = 205$. The 2 convolutional layers have 4 filters each and swish activation. The GRU has $\tanh$ activation and is depicted as unrolled with a layer per time-step.
• Figure 5: Gas release in polycrystalline systems. Upper panel: initial state, red dots represent bubbles in voids. Lower panel: gas concentration at $t=0.0005$.