Cross-user activity recognition via temporal relation optimal transport

TL;DR

This work tackles cross-user HAR under non-i.i.d. conditions by introducing Temporal Relation Optimal Transport (TROT), which integrates a Hidden Markov Model–based temporal relation representation with Optimal Transport–based data distribution alignment and a temporal-order regularizer. By extracting temporal sub-activity states as Gaussians and aligning their centers across users via a learned coupling, TROT enables a barycentric mapping that transfers source-domain data to the target domain. The method is solved with a generalized conditional gradient approach and evaluated on OPPT, PAMAP2, and DSADS, where it consistently outperforms state-of-the-art DA baselines. The results demonstrate that incorporating temporal structure into domain alignment yields robust cross-user HAR performance, with implications for real-world, heterogeneous sensing systems; future work includes relaxing the Markov assumption and quantifying user-distance to guide applicability.

Abstract

Current research on human activity recognition (HAR) mainly assumes that training and testing data are drawn from the same distribution to achieve a generalised model, which means all the data are considered to be independent and identically distributed $\displaystyle (i.i.d.) $. In many real-world applications, this assumption does not hold, and collected training and target testing datasets have non-uniform distribution, such as in the case of cross-user HAR. Domain adaptation is a promising approach for cross-user HAR tasks. Existing domain adaptation works based on the assumption that samples in each domain are $\displaystyle i.i.d. $ and do not consider the knowledge of temporal relation hidden in time series data for aligning data distribution. This strong assumption of $\displaystyle i.i.d. $ may not be suitable for time series-related domain adaptation methods because the samples formed by time series segmentation and feature extraction techniques are only coarse approximations to $\displaystyle i.i.d. $ assumption in each domain. In this paper, we propose the temporal relation optimal transport (TROT) method to utilise temporal relation and relax the $\displaystyle i.i.d. $ assumption for the samples in each domain for accurate and efficient knowledge transfer. We obtain the temporal relation representation and implement temporal relation alignment of activities via the Hidden Markov model (HMM) and optimal transport (OT) techniques. Besides, a new regularisation term that preserves temporal relation order information for an improved optimal transport mapping is proposed to enhance the domain adaptation performance. Comprehensive experiments are conducted on three public activity recognition datasets (i.e. OPPT, PAMAP2 and DSADS), demonstrating that TROT outperforms other state-of-the-art methods.

Figures (3)

• Figure 1: Comparison of different domain adaptation schemes in cross-user HAR.
• Figure 2: TROT Learning Process Figure.
• Figure 3: OT-based Temporal Order Preservation Regularization Figure.