RDumb++: Drift-Aware Continual Test-Time Adaptation

TL;DR

The paper addresses robust continual test-time adaptation (CTTA) under extreme non-stationarity, as exemplified by the CCC benchmark. It introduces RDumb++ as a drift-aware extension of RDumb, coupling two drift scores—entropy-based $z^{(E)}_t$ and KL-divergence-based $z^{(KL)}_t$—with EMA statistics and two reset modes (full and soft). Drift is detected when standardized scores exceed a threshold $k$, triggering the appropriate reset to prevent prediction collapse while preserving useful adaptation. Empirical results on CCC-medium across 9 long streams show RDumb++ yields about 3–5% absolute accuracy gains over RDumb, with full resets providing the strongest and most stable improvements, demonstrating improved long-horizon robustness for CTTA in highly dynamic environments.

Abstract

Continual Test-Time Adaptation (CTTA) seeks to update a pretrained model during deployment using only the incoming, unlabeled data stream. Although prior approaches such as Tent, EATA etc. provide meaningful improvements under short evolving shifts, they struggle when the test distribution changes rapidly or over extremely long horizons. This challenge is exemplified by the CCC benchmark, where models operate over streams of 7.5M samples with continually changing corruption types and severities. We propose RDumb++, a principled extension of RDumb that introduces two drift-detection mechanisms i.e entropy-based drift scoring and KL-divergence drift scoring, together with adaptive reset strategies. These mechanisms allow the model to detect when accumulated adaptation becomes harmful and to recover before prediction collapse occurs. Across CCC-medium with three speeds and three seeds (nine runs, each containing one million samples), RDumb++ consistently surpasses RDumb, yielding approx 3% absolute accuracy gains while maintaining stable adaptation throughout the entire stream. Ablation experiments on drift thresholds and reset strengths further show that drift-aware resetting is essential for preventing collapse and achieving reliable long-horizon CTTA.