DATTA: Domain Diversity Aware Test-Time Adaptation for Dynamic Domain Shift Data Streams

TL;DR

DATTA tackles the challenge of dynamic domain shifts in test-time adaptation by introducing a Domain-Diversity Score to detect single- vs multi-domain batches and three modules—Domain-Diversity Discriminator (DD), Domain-Diversity Adaptive Batch Normalization (DABN), and Domain-Diversity Adaptive Fine-Tuning (DAFT). DD assesses domain diversity using a Domain-Diversity Angle and a KDE-driven threshold to guide adaptation; DABN blends source and test BN statistics according to the domain-diversity score; DAFT gates updates to avoid gradient conflicts. The approach yields substantial performance gains over state-of-the-art TTA methods across CIFAR-10-C, CIFAR-100-C, and ImageNet-C under Single-, Multiple-, and Dynamic-Domain streams, with Avg-All improvements up to 13% and favorable efficiency. The work contributes a principled, real-time mechanism to quantify and respond to domain diversity, enabling robust test-time adaptation in realistic, heterogeneous data streams.

Abstract

Test-Time Adaptation (TTA) addresses domain shifts between training and testing. However, existing methods assume a homogeneous target domain (e.g., single domain) at any given time. They fail to handle the dynamic nature of real-world data, where single-domain and multiple-domain distributions change over time. We identify that performance drops in multiple-domain scenarios are caused by batch normalization errors and gradient conflicts, which hinder adaptation. To solve these challenges, we propose Domain Diversity Adaptive Test-Time Adaptation (DATTA), the first approach to handle TTA under dynamic domain shift data streams. It is guided by a novel domain-diversity score. DATTA has three key components: a domain-diversity discriminator to recognize single- and multiple-domain patterns, domain-diversity adaptive batch normalization to combine source and test-time statistics, and domain-diversity adaptive fine-tuning to resolve gradient conflicts. Extensive experiments show that DATTA significantly outperforms state-of-the-art methods by up to 13%. Code is available at https://github.com/DYW77/DATTA.

Figures (4)

• Figure 1: The top illustrates the Dynamic Domain Shift Data Streams, while the bottom compares classification Top1-accuracy (%) on the CIFAR-100-C dataset using EfficientViT-M5, showing the superior performance of our method (DATTA) over previous TTA methods under dynamic domain shift data streams.
• Figure 2: (a) Impact of number of domain(s) on Accuracy: Illustration of the effect of varying number of domains on accuracy. (b) L2 Norm of Gradient Dynamics: Comparison of the L2 norm of gradient under two different data patterns. Both (a) and (b) use CIFAR-10-C at severity level 5 and ResNet-50. (c) Impact of Only Forward and Fine-Tuning on accuracy. (d) Domain-Diversity angle across two data patterns. Both (c) and (d) use CIFAR-100-C at severity level 5 and ResNet-50.
• Figure 3: Overview. DATTA consists of three modules: Diversity Discriminator takes advantage of an Instance-Normalization-guided projection to capture the data features. Based on the discrimination results, DABN and DAFT conduct an adaptive BN re-correcting and model fine-tuning strategy.
• Figure 4: (a) Illustration of Domain-Diversity Score with growing number of domains by using ResNet-50. (b) Steps vs. Domain-Diversity Score by using ImageNet-C at severity level 5 and ResNet-50. The upper right inset is Domain-Diversity Score in Gaussian KDE Model.

Theorems & Definitions (1)

• Definition 1