Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Knowledge Distillation and Enhanced Subdomain Adaptation Using Graph Convolutional Network for Resource-Constrained Bearing Fault Diagnosis

Mohammadreza Kavianpour, Parisa Kavianpour, Amin Ramezani, Mohammad TH Beheshti

TL;DR

The paper tackles bearing fault diagnosis under changing working conditions with limited labeled data and resource constraints. It introduces KAVI, a unified framework that combines a high-capacity ARMA-filtered GCN teacher with a compact CNN student, powered by Enhanced Local Maximum Mean Discrepancy (ELMMSD) for subdomain alignment and label smoothing to mitigate noisy labels, along with progressive knowledge distillation to transfer robust knowledge to the student. Key contributions include the ELMMSD metric, a three-layer ARMA-GCN teacher, a lightweight student, dynamic SDA/KD training, and a demonstrated 99.67% reduction in model size with only ~2% accuracy loss. Experimental results on CWRU and JNU show superior diagnostic accuracy and substantial computational savings, validated by ablations. The work offers a practical path toward reliable, efficient fault diagnosis on resource-constrained devices in industrial environments, with potential extensions to open-set scenarios and broader domains.

Abstract

Bearing fault diagnosis under varying working conditions faces challenges, including a lack of labeled data, distribution discrepancies, and resource constraints. To address these issues, we propose a progressive knowledge distillation framework that transfers knowledge from a complex teacher model, utilizing a Graph Convolutional Network (GCN) with Autoregressive moving average (ARMA) filters, to a compact and efficient student model. To mitigate distribution discrepancies and labeling uncertainty, we introduce Enhanced Local Maximum Mean Squared Discrepancy (ELMMSD), which leverages mean and variance statistics in the Reproducing Kernel Hilbert Space (RKHS) and incorporates a priori probability distributions between labels. This approach increases the distance between clustering centers, bridges subdomain gaps, and enhances subdomain alignment reliability. Experimental results on benchmark datasets (CWRU and JNU) demonstrate that the proposed method achieves superior diagnostic accuracy while significantly reducing computational costs. Comprehensive ablation studies validate the effectiveness of each component, highlighting the robustness and adaptability of the approach across diverse working conditions.

Knowledge Distillation and Enhanced Subdomain Adaptation Using Graph Convolutional Network for Resource-Constrained Bearing Fault Diagnosis

TL;DR

The paper tackles bearing fault diagnosis under changing working conditions with limited labeled data and resource constraints. It introduces KAVI, a unified framework that combines a high-capacity ARMA-filtered GCN teacher with a compact CNN student, powered by Enhanced Local Maximum Mean Discrepancy (ELMMSD) for subdomain alignment and label smoothing to mitigate noisy labels, along with progressive knowledge distillation to transfer robust knowledge to the student. Key contributions include the ELMMSD metric, a three-layer ARMA-GCN teacher, a lightweight student, dynamic SDA/KD training, and a demonstrated 99.67% reduction in model size with only ~2% accuracy loss. Experimental results on CWRU and JNU show superior diagnostic accuracy and substantial computational savings, validated by ablations. The work offers a practical path toward reliable, efficient fault diagnosis on resource-constrained devices in industrial environments, with potential extensions to open-set scenarios and broader domains.

Abstract

Bearing fault diagnosis under varying working conditions faces challenges, including a lack of labeled data, distribution discrepancies, and resource constraints. To address these issues, we propose a progressive knowledge distillation framework that transfers knowledge from a complex teacher model, utilizing a Graph Convolutional Network (GCN) with Autoregressive moving average (ARMA) filters, to a compact and efficient student model. To mitigate distribution discrepancies and labeling uncertainty, we introduce Enhanced Local Maximum Mean Squared Discrepancy (ELMMSD), which leverages mean and variance statistics in the Reproducing Kernel Hilbert Space (RKHS) and incorporates a priori probability distributions between labels. This approach increases the distance between clustering centers, bridges subdomain gaps, and enhances subdomain alignment reliability. Experimental results on benchmark datasets (CWRU and JNU) demonstrate that the proposed method achieves superior diagnostic accuracy while significantly reducing computational costs. Comprehensive ablation studies validate the effectiveness of each component, highlighting the robustness and adaptability of the approach across diverse working conditions.
Paper Structure (31 sections, 32 equations, 4 figures, 6 tables, 1 algorithm)

This paper contains 31 sections, 32 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Bearing Fault Diagnosis
  3. Related Works
  4. Solution and Contribution
  5. Preliminaries
  6. Problem formulation
  7. Graph Convolutional Network with ARMA filter
  8. maximum mean square discrepancy
  9. Proposed Method
  10. Module 1: Teacher and Student Models
  11. Teacher Model Architecture
  12. Student Model Architecture
  13. Subdomain Adaptation Module
  14. Label Smoothing
  15. Definition of Enhanced Local Maximum Mean square Discrepancy
  16. ...and 16 more sections

Figures (4)

  • Figure 1: schematic diagram of proposed method
  • Figure 2: Comparison accuracy of some tasks on diverse baseline for Teacher model
  • Figure 3: Confusion matrix of the KAVI method for task A3$\to$A2
  • Figure 4: Comparison $\mathcal{A}-\text{distance}$ and $\mathcal{A}_{L}-\text{distance}$ for various SDA and DA methods on task J3 $\to$ J1