Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GALA: A GlobAl-LocAl Approach for Multi-Source Active Domain Adaptation

Juepeng Zheng, Peifeng Zhang, Yibin Wen, Qingmei Li, Yang Zhang, Haohuan Fu

TL;DR

The paper tackles Multi-Source Active Domain Adaptation (MS-ADA), where a model must adapt to a target domain with only a handful of labeled examples while leveraging multiple labeled source domains. It introduces GALA, a simple plug-and-play strategy comprising a global step using gradient embeddings and $k$-means clustering, followed by a cluster-wise local step that accounts for domain gaps across sources. The method can be integrated with existing DA frameworks without extra trainable parameters and achieves state-of-the-art results, closely matching fully supervised upper bounds with only 1% target annotations across Digit-Five, Office-Home, and DomainNet. The work provides both empirical evidence and theoretical insight (via $\mathcal{H} \Delta \mathcal{H}$ bounds) into why GALA improves both diversity and transferability in MS-ADA settings, delivering practical gains for real-world cross-domain tasks.

Abstract

Domain Adaptation (DA) provides an effective way to tackle target-domain tasks by leveraging knowledge learned from source domains. Recent studies have extended this paradigm to Multi-Source Domain Adaptation (MSDA), which exploits multiple source domains carrying richer and more diverse transferable information. However, a substantial performance gap still remains between adaptation-based methods and fully supervised learning. In this paper, we explore a more practical and challenging setting, named Multi-Source Active Domain Adaptation (MS-ADA), to further enhance target-domain performance by selectively acquiring annotations from the target domain. The key difficulty of MS-ADA lies in designing selection criteria that can jointly handle inter-class diversity and multi-source domain variation. To address these challenges, we propose a simple yet effective GALA strategy (GALA), which combines a global k-means clustering step for target-domain samples with a cluster-wise local selection criterion, effectively tackling the above two issues in a complementary manner. Our proposed GALA is plug-and-play and can be seamlessly integrated into existing DA frameworks without introducing any additional trainable parameters. Extensive experiments on three standard DA benchmarks demonstrate that GALA consistently outperforms prior active learning and active DA methods, achieving performance comparable to the fully-supervised upperbound while using only 1% of the target annotations.

GALA: A GlobAl-LocAl Approach for Multi-Source Active Domain Adaptation

TL;DR

The paper tackles Multi-Source Active Domain Adaptation (MS-ADA), where a model must adapt to a target domain with only a handful of labeled examples while leveraging multiple labeled source domains. It introduces GALA, a simple plug-and-play strategy comprising a global step using gradient embeddings and -means clustering, followed by a cluster-wise local step that accounts for domain gaps across sources. The method can be integrated with existing DA frameworks without extra trainable parameters and achieves state-of-the-art results, closely matching fully supervised upper bounds with only 1% target annotations across Digit-Five, Office-Home, and DomainNet. The work provides both empirical evidence and theoretical insight (via bounds) into why GALA improves both diversity and transferability in MS-ADA settings, delivering practical gains for real-world cross-domain tasks.

Abstract

Domain Adaptation (DA) provides an effective way to tackle target-domain tasks by leveraging knowledge learned from source domains. Recent studies have extended this paradigm to Multi-Source Domain Adaptation (MSDA), which exploits multiple source domains carrying richer and more diverse transferable information. However, a substantial performance gap still remains between adaptation-based methods and fully supervised learning. In this paper, we explore a more practical and challenging setting, named Multi-Source Active Domain Adaptation (MS-ADA), to further enhance target-domain performance by selectively acquiring annotations from the target domain. The key difficulty of MS-ADA lies in designing selection criteria that can jointly handle inter-class diversity and multi-source domain variation. To address these challenges, we propose a simple yet effective GALA strategy (GALA), which combines a global k-means clustering step for target-domain samples with a cluster-wise local selection criterion, effectively tackling the above two issues in a complementary manner. Our proposed GALA is plug-and-play and can be seamlessly integrated into existing DA frameworks without introducing any additional trainable parameters. Extensive experiments on three standard DA benchmarks demonstrate that GALA consistently outperforms prior active learning and active DA methods, achieving performance comparable to the fully-supervised upperbound while using only 1% of the target annotations.
Paper Structure (22 sections, 4 equations, 9 figures, 6 tables, 1 algorithm)

This paper contains 22 sections, 4 equations, 9 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Domain Adaptation
  4. MSDA
  5. Active Domain Adaptation
  6. Methodology
  7. Problem Setting and Notations
  8. Global Step
  9. Local Step
  10. Embedding GALA into other DA methods
  11. Experiments
  12. Datasets
  13. Training details
  14. Results
  15. Sensitive analysis of $\alpha$
  16. ...and 7 more sections

Figures (9)

  • Figure 1: Different DA and ADA settings. (a) Single-Source Domain Adaptation (SSDA), a standard DA setting with only one source domain. (b) Multi-Source Domain Adaptation (MSDA), with multiple source domains from different styles. (c) Single-Source Active Domain Adaptation (SS-ADA), a standard ADA setting by annotating a few images to address domain shifts according to only one source domain. (d) Multiple-Source Active Domain Adaptation (MS-ADA), by annotating a few images to address domain shifts according to multiple source domains with style variations.
  • Figure 2: The accuracy of SOTA DA methods on Digit-Five and DomainNet datasets. Although MSDA methods improve the performance compared to SSDA methods, there still exists considerable accuracy gap between the MSDA methods and the Fully-Supervised (FS) upperbound (with 14.4% and 2.0% for DomainNet and Digit-Five, respectively). However, our GALA significantly increase the accuracy compared to previous DA methods, and is very close to the FS upperbound with only 1% annotation of the target dataset.
  • Figure 3: The framework for our proposed GlobAl-LocAl strategy (GALA). (a) Global step: We select the target queries ($\mathcal{D}_{st}$) to contain both diversity in the gradient embedding space ($g_c\left( x_t^j \right)$) and uncertainty from the perspective of the training model (addressing Challenge 1). (b) Local step: We select the highest value with high uncertainty and large domain gap for each cluster ($\mathcal{C}_s^b$) of $v$ according to Eq. \ref{['eq:v']} (addressing Challenge 2). (c) Other DA methods: Our proposed GALA could be easily plug-and-play to any other DA approaches without any trainable parameters. Pseudo algorithm for GALA is provided in Algorithm \ref{['alg2']}.
  • Figure 4: Examples for each dataset: (a) Digit-Five: Images from MNIST-M (MM), MNIST (MT), SVHN (SV), USPS (UP), and Synthetic Digits (SY). (b) Office-Home: Sample images from Artistic (Ar), Clip-Art (Cl), Product (Pr), and Real-World (Rw). (c) DomainNet: Visuals from Clipart (Cli), Infograph (Inf), Painting (Pai), Quickdraw (Qui), Real (Rea), and Sketch (Ske).
  • Figure 5: Sensitive analysis of $\alpha$ introduced in Sec. \ref{['sec:global']}: (a) Performance on different value of $\alpha$ for Digit-Five (b) Performance on different value of $\alpha$ for Office-Home
  • ...and 4 more figures