Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Contrastive Learning with Auxiliary User Detection for Identifying Activities

Wen Ge, Guanyi Mou, Emmanuel O. Agu, Kyumin Lee

TL;DR

This work expands the contextual scope of the CAHAR task by integrating User Identification within the CAHAR framework, jointly predicting both CAHAR and UI in a new task called User and Context-Aware HAR (UCA-HAR), which enriches personalized and contextual understanding by jointly learning user-invariant and user-specific patterns.

Abstract

Human Activity Recognition (HAR) is essential in ubiquitous computing, with far-reaching real-world applications. While recent SOTA HAR research has demonstrated impressive performance, some key aspects remain under-explored. Firstly, HAR can be both highly contextualized and personalized. However, prior work has predominantly focused on being Context-Aware (CA) while largely ignoring the necessity of being User-Aware (UA). We argue that addressing the impact of innate user action-performing differences is equally crucial as considering external contextual environment settings in HAR tasks. Secondly, being user-aware makes the model acknowledge user discrepancies but does not necessarily guarantee mitigation of these discrepancies, i.e., unified predictions under the same activities. There is a need for a methodology that explicitly enforces closer (different user, same activity) representations. To bridge this gap, we introduce CLAUDIA, a novel framework designed to address these issues. Specifically, we expand the contextual scope of the CA-HAR task by integrating User Identification (UI) within the CA-HAR framework, jointly predicting both CA-HAR and UI in a new task called User and Context-Aware HAR (UCA-HAR). This approach enriches personalized and contextual understanding by jointly learning user-invariant and user-specific patterns. Inspired by SOTA designs in the visual domain, we introduce a supervised contrastive loss objective on instance-instance pairs to enhance model efficacy and improve learned feature quality. Evaluation across three real-world CA-HAR datasets reveals substantial performance enhancements, with average improvements ranging from 5.8% to 14.1% in Matthew's Correlation Coefficient and 3.0% to 7.2% in Macro F1 score.

Contrastive Learning with Auxiliary User Detection for Identifying Activities

TL;DR

This work expands the contextual scope of the CAHAR task by integrating User Identification within the CAHAR framework, jointly predicting both CAHAR and UI in a new task called User and Context-Aware HAR (UCA-HAR), which enriches personalized and contextual understanding by jointly learning user-invariant and user-specific patterns.

Abstract

Human Activity Recognition (HAR) is essential in ubiquitous computing, with far-reaching real-world applications. While recent SOTA HAR research has demonstrated impressive performance, some key aspects remain under-explored. Firstly, HAR can be both highly contextualized and personalized. However, prior work has predominantly focused on being Context-Aware (CA) while largely ignoring the necessity of being User-Aware (UA). We argue that addressing the impact of innate user action-performing differences is equally crucial as considering external contextual environment settings in HAR tasks. Secondly, being user-aware makes the model acknowledge user discrepancies but does not necessarily guarantee mitigation of these discrepancies, i.e., unified predictions under the same activities. There is a need for a methodology that explicitly enforces closer (different user, same activity) representations. To bridge this gap, we introduce CLAUDIA, a novel framework designed to address these issues. Specifically, we expand the contextual scope of the CA-HAR task by integrating User Identification (UI) within the CA-HAR framework, jointly predicting both CA-HAR and UI in a new task called User and Context-Aware HAR (UCA-HAR). This approach enriches personalized and contextual understanding by jointly learning user-invariant and user-specific patterns. Inspired by SOTA designs in the visual domain, we introduce a supervised contrastive loss objective on instance-instance pairs to enhance model efficacy and improve learned feature quality. Evaluation across three real-world CA-HAR datasets reveals substantial performance enhancements, with average improvements ranging from 5.8% to 14.1% in Matthew's Correlation Coefficient and 3.0% to 7.2% in Macro F1 score.

Paper Structure

This paper contains 23 sections, 12 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. CA-HAR Models
  4. Directly Modeling Instance-Label Relationships
  5. Enhanced Auxiliary Modeling of Inter-Label Relationships
  6. User Identification and Applications
  7. Contrastive Loss for HAR
  8. Proposed CLAUDIA Framework
  9. Task Formulation
  10. Model Overview
  11. Objective Function
  12. Experiments
  13. Datasets
  14. Baseline HAR Models
  15. Experimental Setup
  16. ...and 8 more sections

Figures (2)

  • Figure 1: Framework of CLAUDIA.The upper part illustrates the network design, comprising of data encoding and classification. The lower part depicts the objective function, consisting of two types of losses. Our technical innovation lies in two key aspects: 1) the addition of a UI sub-task, thereby broadening the scope of Context-Aware (CA) HAR to effectively include User-Aware (UA) HAR, and 2) a novel approach to improve feature quality by regularizing inter-instance relationships using supervised contrastive loss.
  • Figure 2: Average activity recognition performance of all models across all datasets.