Adaptive Betweenness Clustering for Semi-Supervised Domain Adaptation

TL;DR

This work tackles semi-supervised domain adaptation by enforcing cross-domain semantic transfer to unlabeled target samples through a graph-based framework. It builds a heterogeneous graph linking labeled source/target samples with unlabeled target data, refines connectivity with confidence- and prediction-based pruning, and applies Adaptive Betweenness Clustering to propagate class information across and within domains. The approach achieves state-of-the-art results on DomainNet, Office-Home, and Office-31, with notable gains particularly under challenging domain shifts. The method enhances target-domain generalization by effectively leveraging scarce target labels to create globally categorical domain alignment.

Abstract

Compared to unsupervised domain adaptation, semi-supervised domain adaptation (SSDA) aims to significantly improve the classification performance and generalization capability of the model by leveraging the presence of a small amount of labeled data from the target domain. Several SSDA approaches have been developed to enable semantic-aligned feature confusion between labeled (or pseudo labeled) samples across domains; nevertheless, owing to the scarcity of semantic label information of the target domain, they were arduous to fully realize their potential. In this study, we propose a novel SSDA approach named Graph-based Adaptive Betweenness Clustering (G-ABC) for achieving categorical domain alignment, which enables cross-domain semantic alignment by mandating semantic transfer from labeled data of both the source and target domains to unlabeled target samples. In particular, a heterogeneous graph is initially constructed to reflect the pairwise relationships between labeled samples from both domains and unlabeled ones of the target domain. Then, to degrade the noisy connectivity in the graph, connectivity refinement is conducted by introducing two strategies, namely Confidence Uncertainty based Node Removal and Prediction Dissimilarity based Edge Pruning. Once the graph has been refined, Adaptive Betweenness Clustering is introduced to facilitate semantic transfer by using across-domain betweenness clustering and within-domain betweenness clustering, thereby propagating semantic label information from labeled samples across domains to unlabeled target data. Extensive experiments on three standard benchmark datasets, namely DomainNet, Office-Home, and Office-31, indicated that our method outperforms previous state-of-the-art SSDA approaches, demonstrating the superiority of the proposed G-ABC algorithm.

Figures (19)

• Figure 1: An example to illustrate Adaptive Betweenness Clustering (ABC). The proposed G-ABC algorithm conducts sample clustering between a labeled point (e.g., "B") and an unlabeled point (e.g., "C"), when they have similarity distances within a confidence threshold to the same class prototype (e.g., "A") and they are with similar prediction distributions from the model. Herein, Point "A" guides the clustering process by serving as an intermediary (betweenness) point for Points "B" and "C". Point "D" is outside of the clustering range.
• Figure 2: An overview of the proposed framework and the training loss for Adaptive Betweenness Clustering. Left: (a) Supervision of labeled data from both source and target domains is applied to guarantee partially categorical domain alignment. (b) Within-domain betweenness clustering (WDBC) is used to determine the relationship between labeled and unlabeled target data. (c) Across-domain betweenness clustering (ADBC) is used to effectively align unlabeled target samples with the source domain. D) Auxiliary techniques for model optimization, including self-training, consistency training, etc. These four components together enable globally categorical domain alignment, progressively enhancing the model's performance, with (b) and (c) establishing reliable sample connectivity among training samples, represented by a heterogeneous graph. Right: Given a pairwise label $s_{ij}$ between samples, the training loss of Adaptive Betweenness Clustering aims to bring samples from the same class closer together in the feature space when $s_{ij}=1$, or to separate samples from different classes when $s_{ij}=0$. This allows for semantic transfer from labeled source or target domains to unlabeled target samples. Note that the orange and light-orange samples belong to the same category, namely "plane", while the blue sample is from a different category, i.e. "flower".
• Figure 3: A diagram depicting graph construction and connectivity refinement. (a) demonstrates the initially constructed sample connectivity of the graph, while (d) presents the refined graph after the connectivity refinement process is performed. The connectivity refinement process is further illustrated by (b) and (c), which effectively eliminate noisy connectivity through the CUNR and PDEP strategies, respectively, resulting in a more reliable graph structure to represent the relationships between samples. The technical details of these four subdiagrams are as follows: (a) Using pairwise label similarities between samples, the initial connectivity between training examples in a heterogeneous graph is constructed; (b) Confidence Uncertainty based Node Removal (CUNR) reduces the connectivity towards unreliable unlabeled samples by removing nodes with low predicted confidence; (c) Prediction Dissimilarity based Edge Pruning (PDEP) further removes the connections between graph samples whose probabilistic prediction distributions are dissimilar; (d) A refined graph is obtained to properly capture the pairwise associations between samples.
• Figure 5: The evaluation of Adaptive Betweenness Clustering involves analyzing the confusion matrices for each epoch on every dataset. Each element in these matrices is associated with a Class-wise Similarity Score, denoted as $s(c, c^{\prime})$. This score, defined by Eq. (\ref{['Equation:SccScore']}) in Sec. \ref{['SubSection:Implementation']}, quantifies the similarity between two classes, $c$ and $c^{\prime}$. In this context, class $c$ refers to the class whose samples are the unlabeled target data, while class $c^{\prime}$ includes classes from both the labeled source and target domains. A higher $s(c, c^{\prime})$ score in each element suggests a greater similarity in predictions between the unlabeled samples from class $c$ in the target domain and the labeled source or target data from class $c^{\prime}$. The experiments are performed with (a): "R $\rightarrow$ P" on Office-Home using ResNet-34, and (b): "D$\rightarrow$A" on Office-31 using AlexNet, respectively, both under the 3-shot setup.
• Figure 6: The impact of removing CUNR and PDEP on performance during graph construction. The experiments are performed on adaptation scenarios of "R$\rightarrow$S" on DomainNet, "R$\rightarrow$P" on Office-Home, and "D$\rightarrow$A" on Office-31, respectively. All of them are conducted under a 3-shot setup, with the first two using ResNet-34 as network backbones, while the latter uses AlexNet.