Memory-Efficient Pseudo-Labeling for Online Source-Free Universal Domain Adaptation using a Gaussian Mixture Model

TL;DR

A novel method is proposed that continuously captures the distribution of known classes in the feature space using a Gaussian mixture model (GMM) and allows for the generation of reliable pseudo-labels, and combines a contrastive loss with a KL divergence loss to perform the adaptation.

Abstract

In practice, domain shifts are likely to occur between training and test data, necessitating domain adaptation (DA) to adjust the pre-trained source model to the target domain. Recently, universal domain adaptation (UniDA) has gained attention for addressing the possibility of an additional category (label) shift between the source and target domain. This means new classes can appear in the target data, some source classes may no longer be present, or both at the same time. For practical applicability, UniDA methods must handle both source-free and online scenarios, enabling adaptation without access to the source data and performing batch-wise updates in parallel with prediction. In an online setting, preserving knowledge across batches is crucial. However, existing methods often require substantial memory, which is impractical because memory is limited and valuable, in particular on embedded systems. Therefore, we consider memory-efficiency as an additional constraint. To achieve memory-efficient online source-free universal domain adaptation (SF-UniDA), we propose a novel method that continuously captures the distribution of known classes in the feature space using a Gaussian mixture model (GMM). This approach, combined with entropy-based out-of-distribution detection, allows for the generation of reliable pseudo-labels. Finally, we combine a contrastive loss with a KL divergence loss to perform the adaptation. Our approach not only achieves state-of-the-art results in all experiments on the DomainNet and Office-Home datasets but also significantly outperforms the existing methods on the challenging VisDA-C dataset, setting a new benchmark for online SF-UniDA. Our code is available at https://github.com/pascalschlachter/GMM.

Figures (2)

• Figure 1: Comparison of the memory required for the knowledge transfer between our GMM-based method, a memory queue AdaContrast, and an additional teacher model like used by COMET comet. Our GMM-based method is clearly the most memory-efficient.
• Figure 2: Overview of our proposed GMM-based approach. After pre-training on the source data $\mathcal{D}_\mathrm{s}$, the model is adapted to the target data $\mathcal{D}_\mathrm{t}$ using a combination of the two losses $\mathcal{L}_\mathrm{C}$ and $\mathcal{L}_\mathrm{KLD}$. For these losses, reliable pseudo-labels are provided by a GMM that models the distributions of the $|\mathcal{Y}_\mathrm{s}|$ known classes in a reduced feature space. Its parameters are iteratively updated with each target batch. To reject samples from new classes as unknown during both pseudo-labeling and prediction, we apply an entropy-based OOD detection.