Uncertainty Quantification for Data-Driven Machine Learning Models in Nuclear Engineering Applications: Where We Are and What Do We Need?

TL;DR

This paper addresses the need for uncertainty quantification in data-driven machine learning applied to nuclear engineering. It contrasts UQ concepts between physics-based and data-driven models, surveys five ML-UQ methodologies (Monte Carlo Dropout, Deep Ensemble, Bayesian Neural Networks, Gaussian Processes, Conformal Prediction), and demonstrates their behavior on both analytical GP benchmarks and SAFARI-1 neutron flux data. The work highlights the variability in uncertainty estimates across methods and argues for a formal verification, validation, and UQ (VVUQ) framework to establish ML credibility in nuclear contexts. By linking SciML with traditional UQ and discussing trustworthiness factors, the paper lays groundwork for standards, benchmarks, and practical adoption of ML in high-consequence nuclear engineering tasks.

Abstract

Machine learning (ML) has been leveraged to tackle a diverse range of tasks in almost all branches of nuclear engineering. Many of the successes in ML applications can be attributed to the recent performance breakthroughs in deep learning, the growing availability of computational power, data, and easy-to-use ML libraries. However, these empirical successes have often outpaced our formal understanding of the ML algorithms. An important but under-rated area is uncertainty quantification (UQ) of ML. ML-based models are subject to approximation uncertainty when they are used to make predictions, due to sources including but not limited to, data noise, data coverage, extrapolation, imperfect model architecture and the stochastic training process. The goal of this paper is to clearly explain and illustrate the importance of UQ of ML. We will elucidate the differences in the basic concepts of UQ of physics-based models and data-driven ML models. Various sources of uncertainties in physical modeling and data-driven modeling will be discussed, demonstrated, and compared. We will also present and demonstrate a few techniques to quantify the ML prediction uncertainties. Finally, we will discuss the need for building a verification, validation and UQ framework to establish ML credibility.

Figures (13)

• Figure 1: Illustration of uncertainty sources in physics-based M&S.
• Figure 2: Illustration of uncertainty sources in data-driven ML models.
• Figure 3: Connections between UQ and SciML.
• Figure 4: Illustration of MCD.
• Figure 5: Illustration of DE.