Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MsFormer: Enabling Robust Predictive Maintenance Services for Industrial Devices

Jiahui Zhou, Dan Li, Ruibing Jin, Jian Lou, Yanran Zhao, Zhenghua Chen, Zigui Jiang, See-Kiong Ng

Abstract

Providing reliable predictive maintenance is a critical industrial AI service essential for ensuring the high availability of manufacturing devices. Existing deep-learning methods present competitive results on such tasks but lack a general service-oriented framework to capture complex dependencies in industrial IoT sensor data. While Transformer-based models show strong sequence modeling capabilities, their direct deployment as robust AI services faces significant bottlenecks. Specifically, streaming sensor data collected in real-world service environments often exhibits multi-scale temporal correlations driven by machine working principles. Besides, the datasets available for training time-to-failure predictive services are typically limited in size. These issues pose significant challenges for directly applying existing models as robust predictive services. To address these challenges, we propose MsFormer, a lightweight Multi-scale Transformer designed as a unified AI service model for reliable industrial predictive maintenance. MsFormer incorporates a Multi-scale Sampling (MS) module and a tailored position encoding mechanism to capture sequential correlations across multi-streaming service data. Additionally, to accommodate data-scarce service environments, MsFormer adopts a lightweight attention mechanism with straightforward pooling operations instead of self-attention. Extensive experiments on real-world datasets demonstrate that the proposed framework achieves significant performance improvements over state-of-the-art methods. Furthermore, MsFormer outperforms across industrial devices and operating conditions, demonstrating strong generalizability while maintaining a highly reliable Quality of Service (QoS).

MsFormer: Enabling Robust Predictive Maintenance Services for Industrial Devices

Abstract

Providing reliable predictive maintenance is a critical industrial AI service essential for ensuring the high availability of manufacturing devices. Existing deep-learning methods present competitive results on such tasks but lack a general service-oriented framework to capture complex dependencies in industrial IoT sensor data. While Transformer-based models show strong sequence modeling capabilities, their direct deployment as robust AI services faces significant bottlenecks. Specifically, streaming sensor data collected in real-world service environments often exhibits multi-scale temporal correlations driven by machine working principles. Besides, the datasets available for training time-to-failure predictive services are typically limited in size. These issues pose significant challenges for directly applying existing models as robust predictive services. To address these challenges, we propose MsFormer, a lightweight Multi-scale Transformer designed as a unified AI service model for reliable industrial predictive maintenance. MsFormer incorporates a Multi-scale Sampling (MS) module and a tailored position encoding mechanism to capture sequential correlations across multi-streaming service data. Additionally, to accommodate data-scarce service environments, MsFormer adopts a lightweight attention mechanism with straightforward pooling operations instead of self-attention. Extensive experiments on real-world datasets demonstrate that the proposed framework achieves significant performance improvements over state-of-the-art methods. Furthermore, MsFormer outperforms across industrial devices and operating conditions, demonstrating strong generalizability while maintaining a highly reliable Quality of Service (QoS).
Paper Structure (26 sections, 11 equations, 6 figures, 7 tables)

This paper contains 26 sections, 11 equations, 6 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Transformer Models in Terms of Feature Extraction
  4. Transformer Models in Terms of Tailored Architectures
  5. Methodology
  6. Overall Architecture
  7. Multi-scale Sampling Module
  8. Structure of Transformer Blocks
  9. Lightweight Attention Mechainism
  10. Relative Position Encoding Block
  11. Multi-scale Position Encoding Mechanism
  12. Experiments
  13. Dataset and Experimental Settings
  14. Datasets
  15. Experimental Setting
  16. ...and 11 more sections

Figures (6)

  • Figure 1: The overall architecture of the MsFormer.
  • Figure 2: The illustration for the multi-scale position encoding mechanism.
  • Figure 3: Performance comparison with fixed dilation factor $\lambda_w$ in the MS module.
  • Figure 4: Results of various dilation factors $\lambda_w$ in the MS module on FD002.
  • Figure 5: Sensitivity analysis for parameter $c_1$ in position encoding mechanisms.
  • ...and 1 more figures