Cross-user activity recognition using deep domain adaptation with temporal relation information

TL;DR

This work tackles cross-user HAR under out-of-distribution conditions by introducing Deep Temporal State Domain Adaptation (DTSDA), a time-series aware domain adaptation framework. DTSDA explicitly models temporal relations through the novel concepts of Temporal State and Temporal Consistency, and employs adversarial learning across three iterative components to learn user-invariant temporal representations and robust activity classifiers. A key contribution is Pseudo Temporal State Labeling, a self-supervised mechanism that enforces temporal coherence and guides cross-user alignment without requiring target labels. Empirical results on three public HAR datasets show that DTSDA consistently outperforms baselines, highlighting the practical impact of leveraging temporal structure for cross-user HAR in sensor-rich environments.

Abstract

Human Activity Recognition (HAR) is a cornerstone of ubiquitous computing, with promising applications in diverse fields such as health monitoring and ambient assisted living. Despite significant advancements, sensor-based HAR methods often operate under the assumption that training and testing data have identical distributions. However, in many real-world scenarios, particularly in sensor-based HAR, this assumption is invalidated by out-of-distribution ($\displaystyle o.o.d.$) challenges, including differences from heterogeneous sensors, change over time, and individual behavioural variability. This paper centres on the latter, exploring the cross-user HAR problem where behavioural variability across individuals results in differing data distributions. To address this challenge, we introduce the Deep Temporal State Domain Adaptation (DTSDA) model, an innovative approach tailored for time series domain adaptation in cross-user HAR. Contrary to the common assumption of sample independence in existing domain adaptation approaches, DTSDA recognizes and harnesses the inherent temporal relations in the data. Therefore, we introduce 'Temporal State', a concept that defined the different sub-activities within an activity, consistent across different users. We ensure these sub-activities follow a logical time sequence through 'Temporal Consistency' property and propose the 'Pseudo Temporal State Labeling' method to identify the user-invariant temporal relations. Moreover, the design principle of DTSDA integrates adversarial learning for better domain adaptation. Comprehensive evaluations on three HAR datasets demonstrate DTSDA's superior performance in cross-user HAR applications by briding individual behavioral variability using temporal relations across sub-activities.

Figures (14)

• Figure 1: The design principle overview of DTSDA.
• Figure 2: The method overview of DTSDA.
• Figure 3: The data distribution adjustment overview of DTSDA.
• Figure 4: An example of pseudo class temporal state label .
• Figure 5: The network architecture of feature extractor module.