Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition

TL;DR

This work tackles cross-person HAR under distribution shifts by proposing Optimization-Free Test-Time Adaptation (OFTTA), which jointly adjusts the feature extractor and classifier without gradient-based optimization. The feature extractor uses Exponential Decay Test-time Normalization (EDTN) to blend Test-time BN and Conventional BN across layers, while the classifier relies on a memory-based prototype approach updated via pseudo labels. Empirical results across three public HAR datasets and two TTA settings show that OFTTA outperforms optimization-free baselines and remains competitive with optimization-based methods, with demonstrated efficiency on edge devices. The approach offers a practical, hardware-friendly solution for real-world HAR systems facing subject- and distribution-driven covariate shifts.

Abstract

Human Activity Recognition (HAR) models often suffer from performance degradation in real-world applications due to distribution shifts in activity patterns across individuals. Test-Time Adaptation (TTA) is an emerging learning paradigm that aims to utilize the test stream to adjust predictions in real-time inference, which has not been explored in HAR before. However, the high computational cost of optimization-based TTA algorithms makes it intractable to run on resource-constrained edge devices. In this paper, we propose an Optimization-Free Test-Time Adaptation (OFTTA) framework for sensor-based HAR. OFTTA adjusts the feature extractor and linear classifier simultaneously in an optimization-free manner. For the feature extractor, we propose Exponential DecayTest-time Normalization (EDTN) to replace the conventional batch normalization (CBN) layers. EDTN combines CBN and Test-time batch Normalization (TBN) to extract reliable features against domain shifts with TBN's influence decreasing exponentially in deeper layers. For the classifier, we adjust the prediction by computing the distance between the feature and the prototype, which is calculated by a maintained support set. In addition, the update of the support set is based on the pseudo label, which can benefit from reliable features extracted by EDTN. Extensive experiments on three public cross-person HAR datasets and two different TTA settings demonstrate that OFTTA outperforms the state-of-the-art TTA approaches in both classification performance and computational efficiency. Finally, we verify the superiority of our proposed OFTTA on edge devices, indicating possible deployment in real applications. Our code is available at https://github.com/Claydon-Wang/OFTTA.

Figures (15)

• Figure 1: T-SNE visualization in the domain space using UniMiB. Each domain is denoted by a specific color.
• Figure 2: Classification comparison between random splitting and the subject-wise splitting strategy.
• Figure 3: Incremental memory and time cost brought by TENT optimization.
• Figure 4: The visualization of different batch statistics, where ${{\mu }_{s}}$ and ${{\sigma }_{s}}$ are the mean and variance of the source statistics. ${{\mu }_{t}}$ and ${{\sigma }_{t}}$ are the mean and variance of the statistics of the test batch. Our EDTN combines TBN and CBN in an exponential decay manner, where CBN accounts for a larger proportion in the deep layer.
• Figure 5: Overview of the proposed OFTTA framework.

Theorems & Definitions (1)

• definition 1: Test-time adaptation for HAR