Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TSAK: Two-Stage Semantic-Aware Knowledge Distillation for Efficient Wearable Modality and Model Optimization in Manufacturing Lines

Hymalai Bello, Daniel Geißler, Sungho Suh, Bo Zhou, Paul Lukowicz

TL;DR

This paper tackles the challenge of deploying accurate wearable HAR in manufacturing lines under tight resource constraints. It introduces TSAK, a two-stage semantic-aware knowledge distillation framework that transfers rich multimodal teacher knowledge into a lightweight student that relies on a 3-axis hand accelerometer, using Attn-Rep, Causal-Rep, and Combi-Rep representations guided by a semantic classifier. Evaluations on Smart Factory and OpenPack datasets show that the TSAK-distilled student is 79% smaller, runs 8.88× faster, and requires 96.6% less FLOPS than the teacher, with SC-Logit KD delivering the strongest F1-score improvements (up to +10.5% on OpenPack). These results demonstrate the feasibility of efficient, privacy-aware wearable HAR suitable for real-world industrial deployment, enabling broader adoption of human-robot collaboration in manufacturing.

Abstract

Smaller machine learning models, with less complex architectures and sensor inputs, can benefit wearable sensor-based human activity recognition (HAR) systems in many ways, from complexity and cost to battery life. In the specific case of smart factories, optimizing human-robot collaboration hinges on the implementation of cutting-edge, human-centric AI systems. To this end, workers' activity recognition enables accurate quantification of performance metrics, improving efficiency holistically. We present a two-stage semantic-aware knowledge distillation (KD) approach, TSAK, for efficient, privacy-aware, and wearable HAR in manufacturing lines, which reduces the input sensor modalities as well as the machine learning model size, while reaching similar recognition performance as a larger multi-modal and multi-positional teacher model. The first stage incorporates a teacher classifier model encoding attention, causal, and combined representations. The second stage encompasses a semantic classifier merging the three representations from the first stage. To evaluate TSAK, we recorded a multi-modal dataset at a smart factory testbed with wearable and privacy-aware sensors (IMU and capacitive) located on both workers' hands. In addition, we evaluated our approach on OpenPack, the only available open dataset mimicking the wearable sensor placements on both hands in the manufacturing HAR scenario. We compared several KD strategies with different representations to regulate the training process of a smaller student model. Compared to the larger teacher model, the student model takes fewer sensor channels from a single hand, has 79% fewer parameters, runs 8.88 times faster, and requires 96.6% less computing power (FLOPS).

TSAK: Two-Stage Semantic-Aware Knowledge Distillation for Efficient Wearable Modality and Model Optimization in Manufacturing Lines

TL;DR

This paper tackles the challenge of deploying accurate wearable HAR in manufacturing lines under tight resource constraints. It introduces TSAK, a two-stage semantic-aware knowledge distillation framework that transfers rich multimodal teacher knowledge into a lightweight student that relies on a 3-axis hand accelerometer, using Attn-Rep, Causal-Rep, and Combi-Rep representations guided by a semantic classifier. Evaluations on Smart Factory and OpenPack datasets show that the TSAK-distilled student is 79% smaller, runs 8.88× faster, and requires 96.6% less FLOPS than the teacher, with SC-Logit KD delivering the strongest F1-score improvements (up to +10.5% on OpenPack). These results demonstrate the feasibility of efficient, privacy-aware wearable HAR suitable for real-world industrial deployment, enabling broader adoption of human-robot collaboration in manufacturing.

Abstract

Smaller machine learning models, with less complex architectures and sensor inputs, can benefit wearable sensor-based human activity recognition (HAR) systems in many ways, from complexity and cost to battery life. In the specific case of smart factories, optimizing human-robot collaboration hinges on the implementation of cutting-edge, human-centric AI systems. To this end, workers' activity recognition enables accurate quantification of performance metrics, improving efficiency holistically. We present a two-stage semantic-aware knowledge distillation (KD) approach, TSAK, for efficient, privacy-aware, and wearable HAR in manufacturing lines, which reduces the input sensor modalities as well as the machine learning model size, while reaching similar recognition performance as a larger multi-modal and multi-positional teacher model. The first stage incorporates a teacher classifier model encoding attention, causal, and combined representations. The second stage encompasses a semantic classifier merging the three representations from the first stage. To evaluate TSAK, we recorded a multi-modal dataset at a smart factory testbed with wearable and privacy-aware sensors (IMU and capacitive) located on both workers' hands. In addition, we evaluated our approach on OpenPack, the only available open dataset mimicking the wearable sensor placements on both hands in the manufacturing HAR scenario. We compared several KD strategies with different representations to regulate the training process of a smaller student model. Compared to the larger teacher model, the student model takes fewer sensor channels from a single hand, has 79% fewer parameters, runs 8.88 times faster, and requires 96.6% less computing power (FLOPS).
Paper Structure (7 sections, 2 equations, 5 figures, 5 tables)

This paper contains 7 sections, 2 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Apparatus and Dataset
  4. Knowledge Distillation Approach
  5. Result and Discussion
  6. Conclusion
  7. Acknowledgements

Figures (5)

  • Figure 1: (A) In the TSAK knowledge distillation (KD) approach, five distillation methods were compared. The cosine similarity loss distills knowledge from one of the hidden vectors at a time; Attention Representation (Attn-Rep), Causal Representation (Causal-Rep), and Combined Representation (Combi-Rep). A shallow classifier is employed to merge and distill knowledge from all the hidden vectors simultaneously, preserving the semantics of the ground truth by logit-based KD (Semantic Classifier). Logit KD is also performed with the teacher's outputs. The output categories are walking, touching screen/buttons (Btn), opening/closing the door (Door), and working inside the factory module (Check). (B) Teacher average results with an F1 score of 85.91% across twelve users.
  • Figure 2: Activities dictionary in the smart factory testbed
  • Figure 3: Twelve users' results in the smart factory scenario with right-handed acceleration channels as input. (A) Baseline; F1 of 75.94% (B) Semantic student; F1 of 81.34%
  • Figure 4: KD-based student comparison with right-handed acceleration inputs. (A) $\alpha$ variations. (B) Logit and SemanticLogit for different temperatures ($alpha=0.99$).
  • Figure 5: Five users' results with OpenPack. (A) Teacher results (12 channels); accelerometer and gyroscope (both hands). (B) Student results with right-handed accelerations as input (Semantic KD) and 10.5% F1 score increase.