Multi-Source Unsupervised Domain Adaptation with Prototype Aggregation

TL;DR

This work tackles multi-source unsupervised domain adaptation by introducing PAMDA, a prototype-based framework that simultaneously models class- and domain-level discrepancies with a group of learned prototypes. It employs a similarity-based transferability weighting to selectively align source prototypes to target features and to utilize only reliable target pseudo-labels for target-prototype construction, thereby mitigating noise and negative transfer. The method optimizes a dual objective that combines source supervision with prototype-guided alignment, formalized through class- and domain-prototype discrepancies $D_c$ and $D_d$, and supported by a theoretical bound on target error that is reduced as $L_{cls}$ and $D$ are minimized. Empirically, PAMDA achieves state-of-the-art or competitive performance on three standard MSDA benchmarks (Digits-5, Office-Caltech-10, Office-31), with ablations and visualizations corroborating the effectiveness of class- and domain-level prototype alignment and the robustness to hyperparameter choices.

Abstract

Multi-source domain adaptation (MSDA) plays an important role in industrial model generalization. Recent efforts on MSDA focus on enhancing multi-domain distributional alignment while omitting three issues, e.g., the class-level discrepancy quantification, the unavailability of noisy pseudo-label, and source transferability discrimination, potentially resulting in suboptimal adaption performance. Therefore, we address these issues by proposing a prototype aggregation method that models the discrepancy between source and target domains at the class and domain levels. Our method achieves domain adaptation based on a group of prototypes (i.e., representative feature embeddings). A similarity score-based strategy is designed to quantify the transferability of each domain. At the class level, our method quantifies class-specific cross-domain discrepancy according to reliable target pseudo-labels. At the domain level, our method establishes distributional alignment between noisy pseudo-labeled target samples and the source domain prototypes. Therefore, adaptation at the class and domain levels establishes a complementary mechanism to obtain accurate predictions. The results on three standard benchmarks demonstrate that our method outperforms most state-of-the-art methods. In addition, we provide further elaboration of the proposed method in light of the interpretable results obtained from the analysis experiments.

Figures (5)

• Figure 1: Different transferability of source domains.
• Figure 2: Overview of PAMDA. PAMDA is established on a group of prototypes. (a) Each class prototype is generated according to reliable labels. (b) We design two discrepancy metrics for diverse confidence pseudo-labeled target samples. At the class level, a class-prototype aggregation discrepancy is adopted for class alignment across multiple domains. At the domain level, we adopt a domain-prototype aggregation discrepancy for cross-domain alignment based on a group of domain prototypes (i.e., the mean of all class prototypes in the same domain). (c) Last but not least, we design a source classification loss and a prototype classification loss to drive the model to learn the supervised knowledge from source data and class prototypes, respectively.
• Figure 3: Hyperparameter analysis of $\tau_c$ and $\tau_d$ on Digits-5
• Figure 4: Feature distributions on the “$\rightarrow mm$” task of Digits-5.
• Figure 5: Class weight distributions on the “$\rightarrow mm$” task of Digits-5.