On-demand Test-time Adaptation for Edge Devices

TL;DR

OD-TTA tackles the inefficiency of continual test-time adaptation by triggering adaptation only when domain shift is detected. It combines three innovations: a lightweight domain shift detector using EMA entropy, a source-domain selection mechanism based on BN statistics to start adaptation from a close domain, and a decoupled BN update scheme that updates statistics and affine parameters with different batch sizes to reduce memory footprint. Empirical results on CIFAR-10-C, ImageNet-C, and SHIFT show that OD-TTA yields competitive or superior accuracy while lowering energy consumption and memory usage, including support for BN-based models at batch size 1. The approach demonstrates a practical path to make TTA viable on resource-constrained edge devices, with strong potential for deployment on embedded AI systems.

Abstract

Continual Test-time adaptation (CTTA) continuously adapts the deployed model on every incoming batch of data. While achieving optimal accuracy, existing CTTA approaches present poor real-world applicability on resource-constrained edge devices, due to the substantial memory overhead and energy consumption. In this work, we first introduce a novel paradigm -- on-demand TTA -- which triggers adaptation only when a significant domain shift is detected. Then, we present OD-TTA, an on-demand TTA framework for accurate and efficient adaptation on edge devices. OD-TTA comprises three innovative techniques: 1) a lightweight domain shift detection mechanism to activate TTA only when it is needed, drastically reducing the overall computation overhead, 2) a source domain selection module that chooses an appropriate source model for adaptation, ensuring high and robust accuracy, 3) a decoupled Batch Normalization (BN) update scheme to enable memory-efficient adaptation with small batch sizes. Extensive experiments show that OD-TTA achieves comparable and even better performance while reducing the energy and computation overhead remarkably, making TTA a practical reality.

Figures (13)

• Figure 1: OD-TTA achieves a superior trade-off between memory, energy, and accuracy compared to state-of-the-art CTTA baselines. The radius of circles represents memory usage (See \ref{['sec:appendix_visualization']}).
• Figure 2: OD-TTA overview. The model performs regular inference while monitoring domain shifts. Once a shift is detected, OD-TTA selects the most similar BN candidate from a candidate pool and asynchronously adapts the BN statistics and affine parameters using a few new domain data.
• Figure 3: (a) Candidate pool construction; (b) storage comparison of saving only BN layers and the full model.
• Figure 4: Energy consumption for processing domain data sequences of varying lengths under batch size = (a) 1 and (b) 16.
• Figure 5: EMA entropy change along data stream on Cifar10-C. The red dotted lines are where domain shift is detected. Domains change after every 10,000 samples, as denoted by the changes in background color, which visually highlight transitions between domains.