Uncertainty-guided Open-Set Source-Free Unsupervised Domain Adaptation with Target-private Class Segregation

TL;DR

This work tackles Source-Free Open-set UDA (SF-OSDA), where no source data is accessible during adaptation and the target domain contains novel classes. It introduces a granular approach to target-private classes by extending the classifier with private prototypes $W_P$ alongside shared prototypes $W_S$, initialized via clustering on target features and aligned to source prototypes. Pseudo-labels are progressively refined through neighbours-consensus and uncertainty-based sample selection, while learning is regularized by a novel NL-InfoNCELoss that integrates negative learning into contrastive learning, improving robustness to noisy pseudo-labels. The method achieves state-of-the-art performance on Office31 and Office-Home SF-OSDA benchmarks and reveals the underlying semantics of target-private classes, enabling potential novel class discovery with a well-structured feature space.

Abstract

Standard Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from a labeled source domain to an unlabeled target but usually requires simultaneous access to both source and target data. Moreover, UDA approaches commonly assume that source and target domains share the same labels space. Yet, these two assumptions are hardly satisfied in real-world scenarios. This paper considers the more challenging Source-Free Open-set Domain Adaptation (SF-OSDA) setting, where both assumptions are dropped. We propose a novel approach for SF-OSDA that exploits the granularity of target-private categories by segregating their samples into multiple unknown classes. Starting from an initial clustering-based assignment, our method progressively improves the segregation of target-private samples by refining their pseudo-labels with the guide of an uncertainty-based sample selection module. Additionally, we propose a novel contrastive loss, named NL-InfoNCELoss, that, integrating negative learning into self-supervised contrastive learning, enhances the model robustness to noisy pseudo-labels. Extensive experiments on benchmark datasets demonstrate the superiority of the proposed method over existing approaches, establishing new state-of-the-art performance. Notably, additional analyses show that our method is able to learn the underlying semantics of novel classes, opening the possibility to perform novel class discovery.

Figures (6)

• Figure 1: Overview of the proposed adaptation approach. (a) Target samples are clustered based on the features extracted from the pretrained source model, to provide an initial pseudo-labels assignment. Next, clustering allows the identification of class centroids (coloured shapes), which are matched against the most similar prototypes of shared classes. Such prototypes are taken as the columns of weight matrix $W$ of the classifier (blue clusters). Since the target domain has more classes than the source domain, some clusters are left out from this matching, which will be treated as target-private classes (yellow clusters). (b) After the refining of the pseudo-label, its uncertainty is estimated using two approaches. Neighbours consensus: the uncertainty is determined by analysing the consensus of neighbours on the refined pseudo-label. Class separation: the model uncertainty is estimated based on the distance w.r.t. the two closest class prototypes. A novel contrastive loss (NL-InfoNCELoss) gathers same-class samples in the features space, while being robust to noisy pseudo-labels.
• Figure 2: The extended source model $h_t$ incorrectly classifies samples from private classes as belonging to shared ones. Nonetheless, samples from both shared and private classes exhibit a low intra-class variability. Building on this observation, we cluster the feature space to perform an initial pseudo-labels assignment on the target domain. The figure shows the initial identification of target-private samples using clustering. In (a) and (b) coloured samples belong to target-private classes, while gray samples belongs to shared classes. The colour of samples represents the predicted class. For visualisation purposes, all samples predicted in a target-private class are represented in the blue class. The $\bullet$ symbol represents correct predictions, whereas the $\times$ represents incorrect predictions. While the source model cannot correctly assign target-private samples, the clustering leverages the structure in the features space to aggregate private samples and provides a good pseudo-labels initialisation. In fact, after the clustering, more target-private samples are correctly classified in the blue class. Nonetheless, pseudo-labels still contain noise, which is progressively reduced during training.
• Figure 3: Pseudo-label uncertainty via Class Separation. The uncertainty of the pseudo-labels is estimated by computing the distance of samples from the two closest class prototypes. If a sample is similarly distant in the features space from two classes, its pseudo-label has high uncertainty. Contrarily, if a sample is close to one class and far from the other, its pseudo-label has low uncertainty and we consider it as reliable.
• Figure 4: A couple of target images would be wrongly considered as a negative pair if only comparing the latest predictions (green box). Instead, since $x_t^i$ and $x_t^j$ share the same pseudo-labels (at least once) in the past T epochs, i.e. $\{e-T,...,e-1 \}$, we exclude them from the list of negative pairs. Figure from Litrico, reproduced with the permission of the authors.
• Figure 5: $HOS$ score on three sub tasks of Office31 ($\%$) with a various number of estimated target classes $|C_P|$. The best performance is obtained with $|C_P| = |C_S|$ target classes.