A temporal scale transformer framework for precise remaining useful life prediction in fuel cells

TL;DR

The paper tackles remaining useful life prediction for PEMFCs by extending the Transformer paradigm with a Temporal Scale Transformer (TSTransformer). Building on the iTransformer, it introduces multi-scale attention over temporally reduced Key/Value matrices via Conv1D to capture both global trends and local details in multivariate time series, enabling accurate and efficient RUL forecasts. Through ageing tests on PEMFC stacks and a rigorous evaluation framework, the TSTransformer achieves superior performance (e.g., Score$_{RUL}$ = 0.914 and RMSE = 0.0033) and excellent accuracy at critical fault thresholds, outperforming LSTM, Transformer, and iTransformer baselines. The results indicate strong potential for data-driven prognostics in renewable energy, with practical implications for predictive maintenance and reliability in PEMFC-based systems, and point to future work on real-world deployment and online transfer learning.

Abstract

In exploring Predictive Health Management (PHM) strategies for Proton Exchange Membrane Fuel Cells (PEMFC), the Transformer model, widely used in data-driven approaches, excels in many fields but struggles with time series analysis due to its self-attention mechanism, which yields a complexity of the input sequence squared and low computational efficiency. It also faces challenges in capturing both global long-term dependencies and local details effectively. To tackle this, we propose the Temporal Scale Transformer (TSTransformer), an enhanced version of the inverted Transformer (iTransformer). Unlike traditional Transformers that treat each timestep as an input token, TSTransformer maps sequences of varying lengths into tokens at different stages for inter-sequence modeling, using attention to capture multivariate correlations and feed-forward networks (FFN) to encode sequence representations. By integrating a one-dimensional convolutional layer into the multivariate attention for multi-level scaling of K and V matrices, it improves local feature extraction, captures temporal scale characteristics, and reduces token count and computational costs. Experiments comparing TSTransformer with models like Long Short-Term Memory, iTransformer, and Transformer demonstrate its potential as a powerful tool for advancing PHM in renewable energy, effectively addressing the limitations of pure Transformer models in data-driven time series tasks.

Figures (7)

• Figure 1: Mechanism structure of a single PEMFC.
• Figure 2: Visualisation of original data, condensed data and filtered data.
• Figure 3: Overall architecture of Temporal scale Transformer(TSTransformer).
• Figure 4: Condensed Overview of the TSTransformer: (a) Initiates with embedding, transforming time series data into discrete, variate-specific tokens. (b) Implements a convolution-based attention mechanism, adjusting the scale of K and V matrices for diverse temporal resolutions. (c) Applies self-attention to the tokens, highlighting inter-variate correlations and enhancing interpretability. (d) This module uses residual connections and normalisation to harmonise outputs from the attention and feed-forward layers. This ensures a balanced and consistent representation of variables. (e) It concludes with a shared FFN that refines token representations for succinct forecasting.
• Figure 5: Prediction results of the TSTransformer model.