Evaluating deep learning models for fault diagnosis of a rotating machinery with epistemic and aleatoric uncertainty

TL;DR

Uncertainty-aware deep learning enhances fault diagnosis for rotating machinery by distinguishing in-distribution from out-of-distribution data under epistemic and aleatoric uncertainty. The authors compare MC dropout, Bayesian neural networks, and deep ensembles on the CWRU bearing dataset, introducing two entropy-based thresholds $\tau_1$ and $\tau_2$ to detect OOD data. Deep ensembles, particularly De2, yield the strongest OOD detection across uncertainty regimes and maintain favorable inference times, while non ensemble methods show weaker performance under noise. The study provides practical guidelines on threshold selection and noise handling for deploying trustworthy fault-diagnosis systems in Industry 4.0 contexts.

Abstract

Uncertainty-aware deep learning (DL) models recently gained attention in fault diagnosis as a way to promote the reliable detection of faults when out-of-distribution (OOD) data arise from unseen faults (epistemic uncertainty) or the presence of noise (aleatoric uncertainty). In this paper, we present the first comprehensive comparative study of state-of-the-art uncertainty-aware DL architectures for fault diagnosis in rotating machinery, where different scenarios affected by epistemic uncertainty and different types of aleatoric uncertainty are investigated. The selected architectures include sampling by dropout, Bayesian neural networks, and deep ensembles. Moreover, to distinguish between in-distribution and OOD data in the different scenarios two uncertainty thresholds, one of which is introduced in this paper, are alternatively applied. Our empirical findings offer guidance to practitioners and researchers who have to deploy real-world uncertainty-aware fault diagnosis systems. In particular, they reveal that, in the presence of epistemic uncertainty, all DL models are capable of effectively detecting, on average, a substantial portion of OOD data across all the scenarios. However, deep ensemble models show superior performance, independently of the uncertainty threshold used for discrimination. In the presence of aleatoric uncertainty, the noise level plays an important role. Specifically, low noise levels hinder the models' ability to effectively detect OOD data. Even in this case, however, deep ensemble models exhibit a milder degradation in performance, dominating the others. These achievements, combined with their shorter inference time, make deep ensemble architectures the preferred choice.

Figures (20)

• Figure 1: Representation of epistemic and aleatoric uncertainty for a classification problem.
• Figure 2: Dropout sampling architecture: $K$ different models, obtained by $K$ dropout samplings, are used to generate $K$ outputs for a given example.
• Figure 3: Monte Carlo dropout approaches: Gaussian (a) and Bernoulli (b) Dropout; Gaussian (c) and Bernoulli (d) DropConnect; Spike-and-Slab Sampling (e).
• Figure 4: Bayesian neural network architecture: each weight in the network is represented by a random variable with a given distribution.
• Figure 5: Deep ensemble architecture: $K$ different base learners are used to generate $K$ outputs for a given example.