Assessment of Uncertainty Quantification in Universal Differential Equations

TL;DR

This work tackles uncertainty quantification in Universal Differential Equations (UDEs), a SciML framework that blends mechanistic models with neural networks. It formalizes a tailored UQ framework for UDEs and compares three epistemic UQ methods—ensembles, variational inference, and MCMC—across three synthetic dynamics problems with Gaussian and Negative Binomial noise. The study finds that ensemble methods and fully Bayesian MCMC generally outperform Variational Inference, though challenges such as multi-modality, threshold selection, and computational cost persist. The results highlight practical trade-offs and motivate hybrid UQ approaches, symmetry-removal strategies, and exploration of model-form uncertainty to improve reliability and applicability of UDEs in complex systems.

Abstract

Scientific Machine Learning is a new class of approaches that integrate physical knowledge and mechanistic models with data-driven techniques for uncovering governing equations of complex processes. Among the available approaches, Universal Differential Equations (UDEs) are used to combine prior knowledge in the form of mechanistic formulations with universal function approximators, like neural networks. Integral to the efficacy of UDEs is the joint estimation of parameters within mechanistic formulations and the universal function approximators using empirical data. The robustness and applicability of resultant models, however, hinge upon the rigorous quantification of uncertainties associated with these parameters, as well as the predictive capabilities of the overall model or its constituent components. With this work, we provide a formalisation of uncertainty quantification (UQ) for UDEs and investigate important frequentist and Bayesian methods. By analysing three synthetic examples of varying complexity, we evaluate the validity and efficiency of ensembles, variational inference and Markov chain Monte Carlo sampling as epistemic UQ methods for UDEs.

Figures (17)

• Figure 1: Overview of the presented uncertainty quantification methods. Given an unknown objective function landscape, we estimate the posterior distribution of the parameters using UDE ensembles, MCMC methods or Variational Inference. Based on random draws from the posterior distribution, predictions of the state trajectory are made, yielding lower and upper bounds for a 99% prediction interval.
• Figure 2: Comparison of the UQ methods on the SEIR Waves problem (Gaussian noise, $\sigma=0.01$).
• Figure 3: Comparison of prediction performance using the same setting as in \ref{['fig:method_comparison']} but with a new initial condition $\bm{x}_0 = (0.8, 0.1, 0.0, 0.1)$.
• Figure S4: Visualization of the values of $\beta$ for the data generation process of the SEIR Pulse and SEIR Waves problem scenarios.
• Figure S2: Waterfall plot for the SEIR Pulse problem with Gaussian noise (0.01) for all models (left), the best 8500 models (middle) and the best 300 models (right) according to the negative log likelihood values (NegLL). The y-values are shifted by the minimal value obtained. The cut-off value clearly discards failed model trainings. Unlike many mechanistic systems, UDEs tend to not converge to a global minimum when using multistart optimization.