SloMo-Fast: Slow-Momentum and Fast-Adaptive Teachers for Source-Free Continual Test-Time Adaptation

TL;DR

The paper tackles CTTA under privacy-sensitive constraints by introducing SloMo-Fast, a source-free dual-teacher framework that combines a Fast-Teacher for rapid domain adaptation and a Slow-Teacher for long-term generalization. It generates class prototypes on-the-fly from unlabeled target data using PLPD-filtered features and guides the Slow-Teacher via contrastive learning, MSE alignment, and information maximization, while the student benefits from symmetric cross-entropy with both teachers. Prediction ensembling, prior correction, and stochastic restoration stabilize continual adaptation, and Cyclic-TTA offers a realistic benchmark where domains recur over time. Empirically, SloMo-Fast achieves state-of-the-art results across 11 CTTA settings on five datasets, including CIFAR10-C, CIFAR100-C, and ImageNet-C, highlighting strong adaptation and retention in cyclic and evolving environments.

Abstract

Continual Test-Time Adaptation (CTTA) is crucial for deploying models in real-world applications with unseen, evolving target domains. Existing CTTA methods, however, often rely on source data or prototypes, limiting their applicability in privacy-sensitive and resource-constrained settings. Additionally, these methods suffer from long-term forgetting, which degrades performance on previously encountered domains as target domains shift. To address these challenges, we propose SloMo-Fast, a source-free, dual-teacher CTTA framework designed for enhanced adaptability and generalization. It includes two complementary teachers: the Slow-Teacher, which exhibits slow forgetting and retains long-term knowledge of previously encountered domains to ensure robust generalization, and the Fast-Teacher rapidly adapts to new domains while accumulating and integrating knowledge across them. This framework preserves knowledge of past domains and adapts efficiently to new ones. We also introduce Cyclic Test-Time Adaptation (Cyclic-TTA), a novel CTTA benchmark that simulates recurring domain shifts. Our extensive experiments demonstrate that SloMo-Fast consistently outperforms state-of-the-art methods across Cyclic-TTA, as well as ten other CTTA settings, highlighting its ability to both adapt and generalize across evolving and revisited domains.

Figures (16)

• Figure 1: Overview of current CTTA approaches utilizing teacher-student models and contrastive learning is on the left. SloMo-Fast, on the right, incorporates a second teacher model to dynamically generate prototypes during testing without requiring source data.
• Figure 2: The SloMo-FAST framework comprises a dual-teacher and student model. The fast teacher $T_1$ quickly adapts to the current domain by taking the exponential moving average of the student. Confident feature vectors from $T_1$ are used to construct robust class prototypes via a priority queue, which refines the slow teacher $T_2$ through contrastive learning. This enables $T_2$ to learn domain-invariant representations while preserving knowledge from previous domains.
• Figure 3: Adaptation rate over a cyclic domain shift. SloMo-Fast shows the best and most stable performance, benefiting from prototype memory and slow-teacher guidance.
• Figure 4: Validation accuracy on the initial domain after sequential adaptation across 15 domains. SloMo-Fast (T2) shows the slowest forgetting, highlighting its ability to retain long-term knowledge while adapting to new domains.
• Figure 5: t-SNE visualization of feature representations and class prototypes (bigger shapes with thick black border). The visualization highlights distinct class separation, showcasing the model's ability to effectively learn discriminative feature representations.