Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Cost-Sensitive Transformer Model for Prognostics Under Highly Imbalanced Industrial Data

Ali Beikmohammadi, Mohammad Hosein Hamian, Neda Khoeyniha, Tony Lindgren, Olof Steinert, Sindri Magnússon

TL;DR

This paper tackles failure prognostics in industrial settings under challenging data conditions by proposing a cost-sensitive Transformer model. It integrates two-stage missing-value handling via feature removal and Bayesian Ridge Imputation, a hybrid resampling scheme (SVM-SMOTE + Repeated ENN), and a Focal Loss-based objective to reflect unequal misclassification costs. Empirical results on the APS dataset from Scania and the SECOM dataset show improved total cost and competitive metrics, with an ablation study confirming the importance of each component. The approach demonstrates practical potential for more reliable and cost-effective predictive maintenance in real-world industrial operations.

Abstract

The rapid influx of data-driven models into the industrial sector has been facilitated by the proliferation of sensor technology, enabling the collection of vast quantities of data. However, leveraging these models for failure detection and prognosis poses significant challenges, including issues like missing values and class imbalances. Moreover, the cost sensitivity associated with industrial operations further complicates the application of conventional models in this context. This paper introduces a novel cost-sensitive transformer model developed as part of a systematic workflow, which also integrates a hybrid resampler and a regression-based imputer. After subjecting our approach to rigorous testing using the APS failure dataset from Scania trucks and the SECOM dataset, we observed a substantial enhancement in performance compared to state-of-the-art methods. Moreover, we conduct an ablation study to analyze the contributions of different components in our proposed method. Our findings highlight the potential of our method in addressing the unique challenges of failure prediction in industrial settings, thereby contributing to enhanced reliability and efficiency in industrial operations.

A Cost-Sensitive Transformer Model for Prognostics Under Highly Imbalanced Industrial Data

TL;DR

This paper tackles failure prognostics in industrial settings under challenging data conditions by proposing a cost-sensitive Transformer model. It integrates two-stage missing-value handling via feature removal and Bayesian Ridge Imputation, a hybrid resampling scheme (SVM-SMOTE + Repeated ENN), and a Focal Loss-based objective to reflect unequal misclassification costs. Empirical results on the APS dataset from Scania and the SECOM dataset show improved total cost and competitive metrics, with an ablation study confirming the importance of each component. The approach demonstrates practical potential for more reliable and cost-effective predictive maintenance in real-world industrial operations.

Abstract

The rapid influx of data-driven models into the industrial sector has been facilitated by the proliferation of sensor technology, enabling the collection of vast quantities of data. However, leveraging these models for failure detection and prognosis poses significant challenges, including issues like missing values and class imbalances. Moreover, the cost sensitivity associated with industrial operations further complicates the application of conventional models in this context. This paper introduces a novel cost-sensitive transformer model developed as part of a systematic workflow, which also integrates a hybrid resampler and a regression-based imputer. After subjecting our approach to rigorous testing using the APS failure dataset from Scania trucks and the SECOM dataset, we observed a substantial enhancement in performance compared to state-of-the-art methods. Moreover, we conduct an ablation study to analyze the contributions of different components in our proposed method. Our findings highlight the potential of our method in addressing the unique challenges of failure prediction in industrial settings, thereby contributing to enhanced reliability and efficiency in industrial operations.
Paper Structure (27 sections, 13 equations, 8 figures, 6 tables, 2 algorithms)

This paper contains 27 sections, 13 equations, 8 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Literature review
  3. Machine Learning in Prognostic Prediction
  4. Dealing with Missing Values Challenge
  5. Dealing with Imbalanced Data Challenge
  6. Dealing with Cost Sensitivity Challenge
  7. Proposed Method
  8. Handling Missing Values
  9. Eliminating Features with a Large Number of Missing Values
  10. Bayesian Ridge Regression-based Imputation for the Remaining Missing Values
  11. Addressing Imbalanced Data
  12. Oversampling by SVM-SMOTE
  13. Undersampling by Repeated ENN
  14. Model Architecture: The Transformer Neural Network
  15. Cost-Sensitive Loss Function Design
  16. ...and 12 more sections

Figures (8)

  • Figure 1: The visual representation of our proposed model's design (a) Overall architecture of our model, (b) Details of the feed forward network, (c) A detailed insight into the proposed MLP module.
  • Figure 2: Air pressure system structure.
  • Figure 3: APS Failure at Scania Trucks dataset distribution.
  • Figure 4: Our proposed data preprocessing workflow.
  • Figure 5: Exploring missing data patterns in the APS Failure at Scania Trucks Dataset during the data preparation process.
  • ...and 3 more figures