A Light-weight Deep Human Activity Recognition Algorithm Using Multi-knowledge Distillation

TL;DR

This article presents Stage-Memory-Logits Distillation (SMLDist), a framework designed to build highly customizable HAR models that achieve optimal performance under various resource constraints, and prioritizes frequency-related features in its distillation process to bolster HAR classification robustness.

Abstract

Inertial sensor-based human activity recognition (HAR) is the base of many human-centered mobile applications. Deep learning-based fine-grained HAR models enable accurate classification in various complex application scenarios. Nevertheless, the large storage and computational overhead of the existing fine-grained deep HAR models hinder their widespread deployment on resource-limited platforms. Inspired by the knowledge distillation's reasonable model compression and potential performance improvement capability, we design a multi-level HAR modeling pipeline called Stage-Logits-Memory Distillation (SMLDist) based on the widely-used MobileNet. By paying more attention to the frequency-related features during the distillation process, the SMLDist improves the HAR classification robustness of the students. We also propose an auto-search mechanism in the heterogeneous classifiers to improve classification performance. Extensive simulation results demonstrate that SMLDist outperforms various state-of-the-art HAR frameworks in accuracy and F1 macro score. The practical evaluation of the Jetson Xavier AGX platform shows that the SMLDist model is both energy-efficient and computation-efficient. These experiments validate the reasonable balance between the robustness and efficiency of the proposed model. The comparative experiments of knowledge distillation on six public datasets also demonstrate that the SMLDist outperforms other advanced knowledge distillation methods of students' performance, which verifies the good generalization of the SMLDist on other classification tasks, including but not limited to HAR.

Figures (15)

• Figure 1: Implicit frequency domain knowledge in HAR tasks. The color of the raw sensor signal demonstrates the Class Activation Map (CAM) of SMLDist models DBLP:conf/cvpr/ZhouKLOT16, where the model focuses on signals with warmer colors and ignores signals with colder colors. The gray bar represents the inherent periodicity of human activities, such as walking periodicity.
• Figure 2: The pipeline of Stage-Memory-Logits Distillation (SMLDist) for HAR.
• Figure 3: Competitive training and automatic search for classifiers. Within the SMLDist pipeline, the gradient adjusts the importance of each classifier, allowing us to select the classifier with the highest importance for deployment in the final model.
• Figure 4: Comparison of Accuracy and F1 macro between SMLDist and other state-of-the-art HAR model architectures, focusing on the generation of users, using the test sets comprising data from users not included in the training set.
• Figure 5: Comparison of Accuracy and F1 macro between SMLDist and other state-of-the-art HAR model architectures, focusing on the generation of sensor displacements, using the test sets comprising self-placement and induced-displacement, which are completely distinct from the ideal-placement in the training set.