Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Agile Multi-Source-Free Domain Adaptation

Xinyao Li, Jingjing Li, Fengling Li, Lei Zhu, Ke Lu

TL;DR

This work tackles Multi-Source-Free Domain Adaptation (MSFDA) by enabling adaptation from multiple source models to an unlabeled target without accessing source data. It introduces Bi-ATEN, a bi-level attention module that learns instance-specific intra-domain weights $alpha^i$ and domain-consistent inter-domain weights $beta$ to assemble cross-domain outputs while keeping source backbones fixed. Bi-ATEN achieves state-of-the-art results on DomainNet with less than $3\%$ of trainable parameters and up to $8\times$ throughput, and can be plugged into existing MSFDA methods to gain additional improvements. Extensive ablations and analyses demonstrate the importance of instance-specific and domain-consistent weighting and show that the approach scales to large backbones, offering a practical pathway for agile MSFDA in real-world deployments.

Abstract

Efficiently utilizing rich knowledge in pretrained models has become a critical topic in the era of large models. This work focuses on adaptively utilizing knowledge from multiple source-pretrained models to an unlabeled target domain without accessing the source data. Despite being a practically useful setting, existing methods require extensive parameter tuning over each source model, which is computationally expensive when facing abundant source domains or larger source models. To address this challenge, we propose a novel approach which is free of the parameter tuning over source backbones. Our technical contribution lies in the Bi-level ATtention ENsemble (Bi-ATEN) module, which learns both intra-domain weights and inter-domain ensemble weights to achieve a fine balance between instance specificity and domain consistency. By slightly tuning source bottlenecks, we achieve comparable or even superior performance on a challenging benchmark DomainNet with less than 3% trained parameters and 8 times of throughput compared with SOTA method. Furthermore, with minor modifications, the proposed module can be easily equipped to existing methods and gain more than 4% performance boost. Code is available at https://github.com/TL-UESTC/Bi-ATEN.

Agile Multi-Source-Free Domain Adaptation

TL;DR

This work tackles Multi-Source-Free Domain Adaptation (MSFDA) by enabling adaptation from multiple source models to an unlabeled target without accessing source data. It introduces Bi-ATEN, a bi-level attention module that learns instance-specific intra-domain weights and domain-consistent inter-domain weights to assemble cross-domain outputs while keeping source backbones fixed. Bi-ATEN achieves state-of-the-art results on DomainNet with less than of trainable parameters and up to throughput, and can be plugged into existing MSFDA methods to gain additional improvements. Extensive ablations and analyses demonstrate the importance of instance-specific and domain-consistent weighting and show that the approach scales to large backbones, offering a practical pathway for agile MSFDA in real-world deployments.

Abstract

Efficiently utilizing rich knowledge in pretrained models has become a critical topic in the era of large models. This work focuses on adaptively utilizing knowledge from multiple source-pretrained models to an unlabeled target domain without accessing the source data. Despite being a practically useful setting, existing methods require extensive parameter tuning over each source model, which is computationally expensive when facing abundant source domains or larger source models. To address this challenge, we propose a novel approach which is free of the parameter tuning over source backbones. Our technical contribution lies in the Bi-level ATtention ENsemble (Bi-ATEN) module, which learns both intra-domain weights and inter-domain ensemble weights to achieve a fine balance between instance specificity and domain consistency. By slightly tuning source bottlenecks, we achieve comparable or even superior performance on a challenging benchmark DomainNet with less than 3% trained parameters and 8 times of throughput compared with SOTA method. Furthermore, with minor modifications, the proposed module can be easily equipped to existing methods and gain more than 4% performance boost. Code is available at https://github.com/TL-UESTC/Bi-ATEN.
Paper Structure (21 sections, 30 equations, 9 figures, 5 tables, 1 algorithm)

This paper contains 21 sections, 30 equations, 9 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Problem Definition
  5. Overview
  6. Bi-level Attention Ensemble
  7. Attention Ensemble as a Pluggable Module
  8. Training Process
  9. Experiments
  10. Datasets and Baselines
  11. Main Results
  12. Analytical Experiments
  13. Conclusion
  14. Acknowledgements
  15. Training Algorithm
  16. ...and 6 more sections

Figures (9)

  • Figure 1: Illustration of instance specificity and domain consistency. Dots are weights assigned to each target sample.
  • Figure 2: Framework of our method. Different colors represent different source domains. For cross-domain outputs, colors on the left semicircles represent domains of bottleneck features while that on the right semicircles represent domains of classifiers that generate the cross-domain output. Best viewed in color.
  • Figure 3: Domain-level inter-domain weight comparison. Bars represent source-only accuracies of source models. Lines represent averaged weights assigned to each source.
  • Figure 4: Class-level inter-domain weight comparison on Office-Home. Bars represent source accuracy. Lines represent weight deviations assigned to each source output.
  • Figure 5: Class-level inter-domain weights on DomainNet. Bars represent source accuracies and lines represent domain weight deviations assigned to each source output.
  • ...and 4 more figures