Generalizable Sensor-Based Activity Recognition via Categorical Concept Invariant Learning

TL;DR

The paper tackles the generalization gap in sensor-based HAR caused by inter-subject distribution shifts. It introduces Categorical Concept Invariant Learning (CCIL), which builds a concept matrix from classifier weights and features to enforce both feature- and logit-invariance, using a CMS loss and a momentum-updated class mean $\hat{\mathbf{M}}_c$. The approach provides a simple objective $\mathcal{L} = \mathcal{L}_{CE} + \alpha \mathcal{L}_{CMS}$ and demonstrates superior cross-domain generalization across four HAR datasets and multiple settings, often outperforming state-of-the-art baselines. This work offers a practical path to robust HAR in real-world deployments without requiring target-domain data during training, and its concept-matrix perspective can inform broader cross-domain time-series learning.

Abstract

Human Activity Recognition (HAR) aims to recognize activities by training models on massive sensor data. In real-world deployment, a crucial aspect of HAR that has been largely overlooked is that the test sets may have different distributions from training sets due to inter-subject variability including age, gender, behavioral habits, etc., which leads to poor generalization performance. One promising solution is to learn domain-invariant representations to enable a model to generalize on an unseen distribution. However, most existing methods only consider the feature-invariance of the penultimate layer for domain-invariant learning, which leads to suboptimal results. In this paper, we propose a Categorical Concept Invariant Learning (CCIL) framework for generalizable activity recognition, which introduces a concept matrix to regularize the model in the training stage by simultaneously concentrating on feature-invariance and logit-invariance. Our key idea is that the concept matrix for samples belonging to the same activity category should be similar. Extensive experiments on four public HAR benchmarks demonstrate that our CCIL substantially outperforms the state-of-the-art approaches under cross-person, cross-dataset, cross-position, and one-person-to-another settings.

Figures (6)

• Figure 1: Domain shift: sensor readings collected from different subjects or different locations of the same subject.
• Figure 2: (a) An overview of our CCIL framework based on the concept matrix. (b) CCIL learns domain-invariant representation by mapping the latent representation of the same-class samples close together
• Figure 3: Visualization of t-SNE embedding for the DSADS dataset. Here, different colors represent different classes. Different shapes indicate different domains. Best viewed in color and zoom in.
• Figure 4: Parameters sensitivity analysis of ${\alpha}$ and ${\lambda}$. The horizontal axis signifies $\alpha \in \{0.1, 0.5, 1, 5, 10\}$, while the vertical axis denotes $\lambda \in \{0, 0.9, 0.99, 0.999, 0.9999\}$.
• Figure 5: Visualized versions of different invariance regularization.