COMET: Contrastive Mean Teacher for Online Source-Free Universal Domain Adaptation

TL;DR

COMET tackles online source-free universal domain adaptation by unifying a mean-teacher pseudo-labeling framework with a contrastive loss to rebuild a discriminative feature space and an entropy-based module to detect unknown classes. The method uses dual entropy thresholds for pseudo-labeling, source-free (COMET-F) or source-prototype (COMET-P) target prototypes, and an entropy loss to stabilize classifier outputs, training online on streaming data. Experiments on DomainNet and VisDA-C across PDA, ODA, and OPDA shifts demonstrate state-of-the-art performance and robustness, establishing a practical baseline for online SF-UniDA. This work enables reliable open-world deployment by gracefully handling domain shift, category shift, and real-time inference, with avenues for continual TTA and zero-shot class handling in the future.

Abstract

In real-world applications, there is often a domain shift from training to test data. This observation resulted in the development of test-time adaptation (TTA). It aims to adapt a pre-trained source model to the test data without requiring access to the source data. Thereby, most existing works are limited to the closed-set assumption, i.e. there is no category shift between source and target domain. We argue that in a realistic open-world setting a category shift can appear in addition to a domain shift. This means, individual source classes may not appear in the target domain anymore, samples of new classes may be part of the target domain or even both at the same time. Moreover, in many real-world scenarios the test data is not accessible all at once but arrives sequentially as a stream of batches demanding an immediate prediction. Hence, TTA must be applied in an online manner. To the best of our knowledge, the combination of these aspects, i.e. online source-free universal domain adaptation (online SF-UniDA), has not been studied yet. In this paper, we introduce a Contrastive Mean Teacher (COMET) tailored to this novel scenario. It applies a contrastive loss to rebuild a feature space where the samples of known classes build distinct clusters and the samples of new classes separate well from them. It is complemented by an entropy loss which ensures that the classifier output has a small entropy for samples of known classes and a large entropy for samples of new classes to be easily detected and rejected as unknown. To provide the losses with reliable pseudo labels, they are embedded into a mean teacher (MT) framework. We evaluate our method across two datasets and all category shifts to set an initial benchmark for online SF-UniDA. Thereby, COMET yields state-of-the-art performance and proves to be consistent and robust across a variety of different scenarios.

Figures (3)

• Figure 1: Overview of COMET. Both the student and teacher are initialized with the weights of the source model. Subsequently, for each target batch $\{x_i^t\}_{i=1}^{N_b}$ first the predictions $\hat{y}_i$ are generated using the output of the student classifier $\boldsymbol{p}_i$ (red box). Second, the pseudo-labels are generated using the classifier output of the teacher $\tilde{\boldsymbol{p}}_i$ (blue box). Third, the pseudo-labels are used to calculate both the contrastive loss $\mathcal{L}_c$ and the entropy loss $\mathcal{L}_e$ (green boxes) to update the student model by backpropagation. Lastly, the teacher weights are updated as an exponential moving average of the student weights (magenta).
• Figure 2: Performance development of all methods for reduced batch sizes shown for the R$\rightarrow$S OPDA scenario.
• Figure 3: Results using different combinations of the losses $\mathcal{L}_c$ and $\mathcal{L}_e$ for the R$\rightarrow$S OPDA scenario. The dashed line indicates the source-only performance.