A Predictive Model Based on Transformer with Statistical Feature Embedding in Manufacturing Sensor Dataset

TL;DR

The paper tackles predictive modeling with limited manufacturing sensor data by introducing a Transformer that leverages statistical feature embedding and window positional encoding to jointly learn sensor-specific and temporal information. The proposed approach achieves strong fault-detection and virtual metrology performance while drastically reducing parameter counts compared with dense DL models, demonstrated on real semiconductor datasets. Key contributions include a novel embedding scheme, a window-aware positional encoding, and comprehensive experiments showing robustness under data scarcity. The findings suggest meaningful practical impact for process monitoring and yield improvement in diverse manufacturing settings, with potential for online learning as more data become available.

Abstract

In the manufacturing process, sensor data collected from equipment is crucial for building predictive models to manage processes and improve productivity. However, in the field, it is challenging to gather sufficient data to build robust models. This study proposes a novel predictive model based on the Transformer, utilizing statistical feature embedding and window positional encoding. Statistical features provide an effective representation of sensor data, and the embedding enables the Transformer to learn both time- and sensor-related information. Window positional encoding captures precise time details from the feature embedding. The model's performance is evaluated in two problems: fault detection and virtual metrology, showing superior results compared to baseline models. This improvement is attributed to the efficient use of parameters, which is particularly beneficial for sensor data that often has limited sample sizes. The results support the model's applicability across various manufacturing industries, demonstrating its potential for enhancing process management and yield.

Figures (10)

• Figure 1: Input embedding of sensor data.
• Figure 2: The Transformer architecture.
• Figure 3: The architecture of the proposed model.
• Figure 4: Statistical feature-based embedding of sensor data.
• Figure 5: Another interpretation of the process of statistical feature-based embedding.