Calibrated Adaptive Teacher for Domain Adaptive Intelligent Fault Diagnosis

TL;DR

This paper proposes a novel method called Calibrated Adaptive Teacher (CAT), where the predictions of the teacher network on target samples throughout the self-training process are calibrated throughout the self-training process, leveraging post hoc calibration techniques.

Abstract

Intelligent Fault Diagnosis (IFD) based on deep learning has proven to be an effective and flexible solution, attracting extensive research. Deep neural networks can learn rich representations from vast amounts of representative labeled data for various applications. In IFD, they achieve high classification performance from signals in an end-to-end manner, without requiring extensive domain knowledge. However, deep learning models usually only perform well on the data distribution they have been trained on. When applied to a different distribution, they may experience performance drops. This is also observed in IFD, where assets are often operated in working conditions different from those in which labeled data have been collected. Unsupervised domain adaptation (UDA) deals with the scenario where labeled data are available in a source domain, and only unlabeled data are available in a target domain, where domains may correspond to operating conditions. Recent methods rely on training with confident pseudo-labels for target samples. However, the confidence-based selection of pseudo-labels is hindered by poorly calibrated confidence estimates in the target domain, primarily due to over-confident predictions, which limits the quality of pseudo-labels and leads to error accumulation. In this paper, we propose a novel UDA method called Calibrated Adaptive Teacher (CAT), where we propose to calibrate the predictions of the teacher network throughout the self-training process, leveraging post-hoc calibration techniques. We evaluate CAT on domain-adaptive IFD and perform extensive experiments on the Paderborn benchmark for bearing fault diagnosis under varying operating conditions. Our proposed method achieves state-of-the-art performance on most transfer tasks.

Figures (12)

• Figure 1: Our proposed Calibrated Adaptive Teacher (CAT). The main novelty involves a post-hoc calibration of the teacher predictions in the target domain throughout the self-training process, improving the quality of pseudo-labels.
• Figure 2: Example of reliability diagram before applying calibration (here, AT on the $0 \rightarrow 1$ task in time domain). The model has higher expected calibration error (ECE) on the target domain than on the source domain.
• Figure 3: Example of reliability diagram after applying Temperature scaling (here, CAT on the $0 \rightarrow 1$ task in time domain). Even though temperature scaling is based on the source validation set, ECE is also drastically reduced on the target domain, owing to well-aligned features.
• Figure 4: Evolution of target pseudo-labels accuracy produced by the teacher network during training for different methods. A boost in accuracy is observed after introduction of the calibration in our proposed CAT. Mean $\pm$ standard deviation over 5 runs with different random seeds.
• Figure 5: Evolution of target accuracy (left) and ECE (right) during training (here, on the $0 \rightarrow 1$ task in time domain). AT significantly improves over the DANN baseline. In addition, our proposed CAT effectively reduces calibration error, leading to an improved accuracy. Mean $\pm$ standard deviation over 5 runs with different random seeds.

Theorems & Definitions (1)

• Definition : Calibrated self-training