Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

P2LHAP:Wearable sensor-based human activity recognition, segmentation and forecast through Patch-to-Label Seq2Seq Transformer

Shuangjian Li, Tao Zhu, Mingxing Nie, Huansheng Ning, Zhenyu Liu, Liming Chen

TL;DR

P2LHAP is introduced, a novel Patch-to-Label Seq2Seq framework that tackles all three tasks in an efficient single-task model and significantly outperforms the state-of-the-art in all three tasks, demonstrating its effectiveness and potential for real-world applications.

Abstract

Traditional deep learning methods struggle to simultaneously segment, recognize, and forecast human activities from sensor data. This limits their usefulness in many fields such as healthcare and assisted living, where real-time understanding of ongoing and upcoming activities is crucial. This paper introduces P2LHAP, a novel Patch-to-Label Seq2Seq framework that tackles all three tasks in a efficient single-task model. P2LHAP divides sensor data streams into a sequence of "patches", served as input tokens, and outputs a sequence of patch-level activity labels including the predicted future activities. A unique smoothing technique based on surrounding patch labels, is proposed to identify activity boundaries accurately. Additionally, P2LHAP learns patch-level representation by sensor signal channel-independent Transformer encoders and decoders. All channels share embedding and Transformer weights across all sequences. Evaluated on three public datasets, P2LHAP significantly outperforms the state-of-the-art in all three tasks, demonstrating its effectiveness and potential for real-world applications.

P2LHAP:Wearable sensor-based human activity recognition, segmentation and forecast through Patch-to-Label Seq2Seq Transformer

TL;DR

P2LHAP is introduced, a novel Patch-to-Label Seq2Seq framework that tackles all three tasks in an efficient single-task model and significantly outperforms the state-of-the-art in all three tasks, demonstrating its effectiveness and potential for real-world applications.

Abstract

Traditional deep learning methods struggle to simultaneously segment, recognize, and forecast human activities from sensor data. This limits their usefulness in many fields such as healthcare and assisted living, where real-time understanding of ongoing and upcoming activities is crucial. This paper introduces P2LHAP, a novel Patch-to-Label Seq2Seq framework that tackles all three tasks in a efficient single-task model. P2LHAP divides sensor data streams into a sequence of "patches", served as input tokens, and outputs a sequence of patch-level activity labels including the predicted future activities. A unique smoothing technique based on surrounding patch labels, is proposed to identify activity boundaries accurately. Additionally, P2LHAP learns patch-level representation by sensor signal channel-independent Transformer encoders and decoders. All channels share embedding and Transformer weights across all sequences. Evaluated on three public datasets, P2LHAP significantly outperforms the state-of-the-art in all three tasks, demonstrating its effectiveness and potential for real-world applications.
Paper Structure (24 sections, 16 equations, 5 figures, 8 tables, 2 algorithms)

This paper contains 24 sections, 16 equations, 5 figures, 8 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Sensor-based activity recognition and segmentation tasks
  4. forecast of activities using wearable sensors
  5. Sensor-based multi-task learning framework
  6. Method
  7. Patch-to-Label Seq2Seq framework
  8. Patching and channel independent Transformer architecture
  9. Patching and channel independent method
  10. Encoder
  11. Decoder
  12. Activity classification
  13. Activity segmentation and smoothing strategy
  14. Activity forecasting
  15. EXPERIMENTS AND RESULTS
  16. ...and 9 more sections

Figures (5)

  • Figure 1: There are different architectures for multitasking approaches as follows: (a) The data is partitioned using sliding windows, followed by separate training of the respective models. These trained models are subsequently utilized for recognition, segmentation, and forecast. (b) Multi-task methods learn models for multiple tasks through a shared encoder and generate separate results for each task. (c) We propose a multi-task P2L Seq2Seq model specifically designed to generate patch-level active label sequences.
  • Figure 2: Overview of the Proposed P2LHAP framework
  • Figure 3: The confusion matrices on the PAMAP2 dataset for patch sizes of 10 and 200 (a) $Patch\_size$=10, (b) $Patch\_size$=200
  • Figure 4: Segment visualization results on PAMAP2 and WISDM datasets
  • Figure 5: Confusion matrix of our model and three other models in predicting activity classification results on the PAMAP2 dataset