Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Re-thinking Human Activity Recognition with Hierarchy-aware Label Relationship Modeling

Jingwei Zuo, Hakim Hacid

TL;DR

This paper addresses human activity recognition (HAR) by incorporating hierarchy-aware label relationships through a graph-based encoding within a flat model. It introduces H-HAR, which combines a predefined and a learnable label graph with an activity data encoder, and trains them with a joint objective that includes label-data alignment, supervised contrastive loss, and cross-entropy loss, i.e., $L = L_{align} + \lambda_{1} L_{con} + \lambda_{2} L_{ce}$. By aligning label and data embeddings in a common representation space, H-HAR yields class-separable embeddings and improved multi-label HAR performance. Experiments on DaLiAc and UCI HAPT demonstrate robustness and improved accuracy over baselines, with potential for integration into more advanced HAR systems and better handling of heterogeneous label relations.

Abstract

Human Activity Recognition (HAR) has been studied for decades, from data collection, learning models, to post-processing and result interpretations. However, the inherent hierarchy in the activities remains relatively under-explored, despite its significant impact on model performance and interpretation. In this paper, we propose H-HAR, by rethinking the HAR tasks from a fresh perspective by delving into their intricate global label relationships. Rather than building multiple classifiers separately for multi-layered activities, we explore the efficacy of a flat model enhanced with graph-based label relationship modeling. Being hierarchy-aware, the graph-based label modeling enhances the fundamental HAR model, by incorporating intricate label relationships into the model. We validate the proposal with a multi-label classifier on complex human activity data. The results highlight the advantages of the proposal, which can be vertically integrated into advanced HAR models to further enhance their performances.

Re-thinking Human Activity Recognition with Hierarchy-aware Label Relationship Modeling

TL;DR

This paper addresses human activity recognition (HAR) by incorporating hierarchy-aware label relationships through a graph-based encoding within a flat model. It introduces H-HAR, which combines a predefined and a learnable label graph with an activity data encoder, and trains them with a joint objective that includes label-data alignment, supervised contrastive loss, and cross-entropy loss, i.e., . By aligning label and data embeddings in a common representation space, H-HAR yields class-separable embeddings and improved multi-label HAR performance. Experiments on DaLiAc and UCI HAPT demonstrate robustness and improved accuracy over baselines, with potential for integration into more advanced HAR systems and better handling of heterogeneous label relations.

Abstract

Human Activity Recognition (HAR) has been studied for decades, from data collection, learning models, to post-processing and result interpretations. However, the inherent hierarchy in the activities remains relatively under-explored, despite its significant impact on model performance and interpretation. In this paper, we propose H-HAR, by rethinking the HAR tasks from a fresh perspective by delving into their intricate global label relationships. Rather than building multiple classifiers separately for multi-layered activities, we explore the efficacy of a flat model enhanced with graph-based label relationship modeling. Being hierarchy-aware, the graph-based label modeling enhances the fundamental HAR model, by incorporating intricate label relationships into the model. We validate the proposal with a multi-label classifier on complex human activity data. The results highlight the advantages of the proposal, which can be vertically integrated into advanced HAR models to further enhance their performances.
Paper Structure (22 sections, 10 equations, 3 figures, 3 tables)

This paper contains 22 sections, 10 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Human Activity Recognition (HAR)
  4. Hierarchical Label Modeling
  5. Problem Formulation
  6. Our proposals
  7. Hierarchy-Aware Label Encoding
  8. Predefined graph structure
  9. Learnable graph structure
  10. Activity Data Encoding
  11. Label-data Joint Embedding Learning
  12. Label-data alignment
  13. Class-separable Embedding Building
  14. Classification and Joint Optimization
  15. Experiments
  16. ...and 7 more sections

Figures (3)

  • Figure 1: The label structure in physical activities, including predefined relationships (solid lines) and implicit relationships (dashed lines).
  • Figure 2: Global system architecture of H-HAR
  • Figure 3: Predefined label hierarchy in DaLiAc and UCI HAPT.

Theorems & Definitions (4)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4