Trust And Balance: Few Trusted Samples Pseudo-Labeling and Temperature Scaled Loss for Effective Source-Free Unsupervised Domain Adaptation

TL;DR

A novel approach marked by two key contributions: Few Trusted Samples Pseudo-labeling (FTSP) and Temperature Scaled Adaptive Loss (TSAL) is introduced, underscoring the effectiveness of the methodology in the SF-UDA landscape.

Abstract

Deep Neural Networks have significantly impacted many computer vision tasks. However, their effectiveness diminishes when test data distribution (target domain) deviates from the one of training data (source domain). In situations where target labels are unavailable and the access to the labeled source domain is restricted due to data privacy or memory constraints, Source-Free Unsupervised Domain Adaptation (SF-UDA) has emerged as a valuable tool. Recognizing the key role of SF-UDA under these constraints, we introduce a novel approach marked by two key contributions: Few Trusted Samples Pseudo-labeling (FTSP) and Temperature Scaled Adaptive Loss (TSAL). FTSP employs a limited subset of trusted samples from the target data to construct a classifier to infer pseudo-labels for the entire domain, showing simplicity and improved accuracy. Simultaneously, TSAL, designed with a unique dual temperature scheduling, adeptly balance diversity, discriminability, and the incorporation of pseudo-labels in the unsupervised adaptation objective. Our methodology, that we name Trust And Balance (TAB) adaptation, is rigorously evaluated on standard datasets like Office31 and Office-Home, and on less common benchmarks such as ImageCLEF-DA and Adaptiope, employing both ResNet50 and ViT-Large architectures. Our results compare favorably with, and in most cases surpass, contemporary state-of-the-art techniques, underscoring the effectiveness of our methodology in the SF-UDA landscape.

Figures (4)

• Figure 1: SF-UDA Pipeline (our contributions in red). In the upper section (a), the source model is trained on the source domain through a conventional supervised method (indicated by the blue arrow). In the lower section (b), adaptation to the target domain is conducted using our proposed pseudo-labeling method (FTSP) and objective function (TSAL), as shown by the yellow arrows. Consistent with the method of liang2020we, the classifier $\gamma$ remains unchanged during the adaptation phase, while the backbone (in green) is adapted.
• Figure 2: Pseudo-labeling with Few Trusted Samples: (a) Classifier trained on the source domain demonstrates robust performance within the same domain. (b) The same classifier underperforms on the unlabeled target domain (represented by grey dots). A minimal set of trusted target samples (indicated by colored dots with white outlines) is selected, being deemed most likely to be correctly classified. (c) Using these few trusted samples, a Multinomial Logistic Regression (MLR) classifier is trained, leading to decision boundaries that align more closely with the target domain and subsequently providing pseudo-labels for the entire target domain. (d) A fraction of uncertain pseudo-labels is eliminated prior to the application of Label Spreading, finalizing the Few Trusted Samples Pseudo-labeling (FTSP) process.
• Figure 3: (a) the temperature scaling in our discrimability term enables a fair competition between model predictions and the pseudo-labels at the end of the training when the network becomes overconfident. (b) our schedules $\tau_{\text{dis}}(\cdot)$ and $\tau_{\text{div}}(\cdot)$ respectively for the discrimanbility and diversity term.
• Figure 4: Discriminability and Diversity terms of the SF-UDA objective averaged across each training epoch with and without temperature scaling. The plot shows the experiment Amazon $\to$ Webcam of Office31.