A Multi-View Consistency Framework with Semi-Supervised Domain Adaptation

TL;DR

A multi-view consistency framework is introduced, which includes two views for training strongly augmented data and a cross-domain affinity learning aimed at aligning features of the same class across different domains, thereby enhancing overall performance.

Abstract

Semi-Supervised Domain Adaptation (SSDA) leverages knowledge from a fully labeled source domain to classify data in a partially labeled target domain. Due to the limited number of labeled samples in the target domain, there can be intrinsic similarity of classes in the feature space, which may result in biased predictions, even when the model is trained on a balanced dataset. To overcome this limitation, we introduce a multi-view consistency framework, which includes two views for training strongly augmented data. One is a debiasing strategy for correcting class-wise prediction probabilities according to the prediction performance of the model. The other involves leveraging pseudo-negative labels derived from the model predictions. Furthermore, we introduce a cross-domain affinity learning aimed at aligning features of the same class across different domains, thereby enhancing overall performance. Experimental results demonstrate that our method outperforms the competing methods on two standard domain adaptation datasets, DomainNet and Office-Home. Combining unsupervised domain adaptation and semi-supervised learning offers indispensable contributions to the industrial sector by enhancing model adaptability, reducing annotation costs, and improving performance.

Figures (7)

• Figure 1: An Illustration of Dataset Composition in SSL, UDA, and SSDA. Circles represent data in the source domain, and triangles represent those in the target domain, which follow different data distributions. Data items in blue are annotated, while others are not. (a) Data in SSL are from a single domain, namely, the target domain only, which is partially annotated; (b) Data in UDA are composed of fully annotated source domain data and unlabeled target domain data. (c) Data in SSDA are composed of fully annotated source domain data and partially annotated target domain data.
• Figure 2: T-SNE Visualization of Features. (a) The class feature distributions of the test data after training the Baseline (S+T) on Office-Home P $\rightarrow$R with 3-shot. Circles represent the sample features in the source domain (Product), and triangles represent the sample features in the target domain (Real), with different colors indicating different classes. (b) For example, the blue triangles represent the Calendar in the target domain, which is confused not only with the Notebook in the target domain, represented by the green triangles but also with the Notebook in the source domain, represented by the green circles.
• Figure 3: Overview of Our Framework. For target unlabeled data, we first generate a weakly augmented view to feed into the model to generate the debiased pseudo-label and the pseudo-negative label, then generate two strongly augmented versions for debiasing, negative, and consistency learning. For both source data and target labeled data, we apply the traditional cross-entropy loss to the prediction after weak augmentation, then select the source domain features with the help of target prototypes for cross-domain affinity learning. Note that we omit supervised learning part for target labeled data.
• Figure 4: An Illustration of Debiasing Learning. Triangles indicate individual samples, with each color representing a distinct class label. Each line illustrates the decision boundary for each class. By debiasing learning, the decision boundaries for the majority classes identified by the model become smaller, and those for minority classes become larger.
• Figure 5: An Illustration of Negative Learning. For an weakly augmented target unlabeled data, the model makes a prediction to generate a pseudo label (Class 2 in this example) and then randomly selects $m$ pseudo-negative labels(Class 1, 6, 7, 8 in this example).