Uncertainty-Aware Pseudo-Label Filtering for Source-Free Unsupervised Domain Adaptation

TL;DR

This work tackles SFUDA by addressing pseudo-label noise without introducing extra models. It introduces Adaptive Pseudo-label Selection (APS) to estimate sample uncertainty via neighbor information and Class-Aware Contrastive Learning (CACL) to mitigate memorization of noisy labels, all within a progressive, coarse-to-fine learning framework (UPA). The method yields state-of-the-art or competitive results across standard SFUDA benchmarks (Office, Office-Home, VisDA-C, DomainNet-126), with ablations confirming the contributions of APS and CACL and visual analyses illustrating reduced domain shift. Overall, UPA offers a simple, robust, and practical solution for source-free adaptation in unlabeled target domains, with code available for reproducibility.

Abstract

Source-free unsupervised domain adaptation (SFUDA) aims to enable the utilization of a pre-trained source model in an unlabeled target domain without access to source data. Self-training is a way to solve SFUDA, where confident target samples are iteratively selected as pseudo-labeled samples to guide target model learning. However, prior heuristic noisy pseudo-label filtering methods all involve introducing extra models, which are sensitive to model assumptions and may introduce additional errors or mislabeling. In this work, we propose a method called Uncertainty-aware Pseudo-label-filtering Adaptation (UPA) to efficiently address this issue in a coarse-to-fine manner. Specially, we first introduce a sample selection module named Adaptive Pseudo-label Selection (APS), which is responsible for filtering noisy pseudo labels. The APS utilizes a simple sample uncertainty estimation method by aggregating knowledge from neighboring samples and confident samples are selected as clean pseudo-labeled. Additionally, we incorporate Class-Aware Contrastive Learning (CACL) to mitigate the memorization of pseudo-label noise by learning robust pair-wise representation supervised by pseudo labels. Through extensive experiments conducted on three widely used benchmarks, we demonstrate that our proposed method achieves competitive performance on par with state-of-the-art SFUDA methods. Code is available at https://github.com/chenxi52/UPA.

Figures (8)

• Figure 1: (a). Evolution of adaptation accuracy (top) and pseudo-label accuracy (bottom) for the SHOTSHOT and SHOT-clean (ideal simulation supervised by manually selected correct pseudo labels) methods on the VisDA-C Visda-c dataset. (b). Feature visualization from 6 interval-selected classes of VisDA-C using t-SNE t-sne, showcasing the differentiation between correct pseudo labels (represented by opaque stars) and incorrect pseudo labels (represented by semi-transparent dots) in the SFUDA setting.
• Figure 2: A comprehensive framework of UPA for addressing SFUDA task. $\mathcal{L}_{IM}$ is applied to all samples. The APS module performs two rounds of uncertainty estimation and samples $\mathbb I_{x_k\in\mathcal{N}_t}[\hat{y}_k = \widetilde{y}_t]$ are useful for $q_t$. $q_t$ is ranked within each class to form a confident sample set $\mathcal{H}$, and then apply $\mathcal{L}_{CE}$ and $\mathcal{L}_{CL}$ to $\mathcal{H}$. Specifically, each sample $x_i$ and its strongly augmented view $x_i'$ are used in the class-wise contrast (represented by the colored matrix block) of $\mathcal{L}_{CL}$.
• Figure 3: The calculation of confidence score $q_t$ with cosine similarity $s$.
• Figure 4: Enhancing data diversity through augmentation techniques: an illustration.
• Figure 5: Parameter sensitivity analysis for SFUDA tasks using UPA on VisDA-C and Office datasets. (a) The confident sample selection ratio $\gamma$; (b) The temperature $\tau$ in $\mathcal{L}_{CL}$; (c) The neighborhood size $K$.