Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Source-Free Domain Adaptation with Diffusion-Guided Source Data Generation

Shivang Chopra, Suraj Kothawade, Houda Aynaou, Aman Chadha

TL;DR

This work tackles Source-Free Domain Adaptation by leveraging diffusion models to generate source-like data guided by target-domain cues and a pre-trained source model. The proposed DM-SFDA framework comprises four phases: selective pseudo labeling of target data, diffusion model fine-tuning via Textual Inversion, source data generation with AlignProp, and unsupervised domain adaptation using diffusion-based domain mixup. Empirical results on Office-31, Office-Home, and VisDA-2017 demonstrate state-of-the-art or competitive SFDA performance, with notable improvements over baselines that do not access source data. The approach offers a privacy-preserving, data-efficient pathway to bridge substantial domain gaps and highlights diffusion models’ potential for practical SFDA applications, while acknowledging computational and scalability considerations for broader deployment.

Abstract

This paper introduces a novel approach to leverage the generalizability of Diffusion Models for Source-Free Domain Adaptation (DM-SFDA). Our proposed DMSFDA method involves fine-tuning a pre-trained text-to-image diffusion model to generate source domain images using features from the target images to guide the diffusion process. Specifically, the pre-trained diffusion model is fine-tuned to generate source samples that minimize entropy and maximize confidence for the pre-trained source model. We then use a diffusion model-based image mixup strategy to bridge the domain gap between the source and target domains. We validate our approach through comprehensive experiments across a range of datasets, including Office-31, Office-Home, and VisDA. The results demonstrate significant improvements in SFDA performance, highlighting the potential of diffusion models in generating contextually relevant, domain-specific images.

Source-Free Domain Adaptation with Diffusion-Guided Source Data Generation

TL;DR

This work tackles Source-Free Domain Adaptation by leveraging diffusion models to generate source-like data guided by target-domain cues and a pre-trained source model. The proposed DM-SFDA framework comprises four phases: selective pseudo labeling of target data, diffusion model fine-tuning via Textual Inversion, source data generation with AlignProp, and unsupervised domain adaptation using diffusion-based domain mixup. Empirical results on Office-31, Office-Home, and VisDA-2017 demonstrate state-of-the-art or competitive SFDA performance, with notable improvements over baselines that do not access source data. The approach offers a privacy-preserving, data-efficient pathway to bridge substantial domain gaps and highlights diffusion models’ potential for practical SFDA applications, while acknowledging computational and scalability considerations for broader deployment.

Abstract

This paper introduces a novel approach to leverage the generalizability of Diffusion Models for Source-Free Domain Adaptation (DM-SFDA). Our proposed DMSFDA method involves fine-tuning a pre-trained text-to-image diffusion model to generate source domain images using features from the target images to guide the diffusion process. Specifically, the pre-trained diffusion model is fine-tuned to generate source samples that minimize entropy and maximize confidence for the pre-trained source model. We then use a diffusion model-based image mixup strategy to bridge the domain gap between the source and target domains. We validate our approach through comprehensive experiments across a range of datasets, including Office-31, Office-Home, and VisDA. The results demonstrate significant improvements in SFDA performance, highlighting the potential of diffusion models in generating contextually relevant, domain-specific images.
Paper Structure (28 sections, 6 equations, 2 figures, 5 tables)

This paper contains 28 sections, 6 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Domain Adaptation
  4. Unsupervised Domain Adaptation
  5. Source-Free Domain Adaptation
  6. DGMs for Domain Adaptation
  7. Preliminaries
  8. Conditional Diffusion Probabilistic Models
  9. Markov Decision Process
  10. Proposed Method
  11. Notations and Problem Definition
  12. Method Overview
  13. Phase I: Selective Pseudo Labeling Target Data
  14. Phase II: Diffusion Model Finetuning on Target Data
  15. Phase III: Source Data Generation using AlignProp
  16. ...and 13 more sections

Figures (2)

  • Figure 1: Overall training pipeline of the proposed DM-SFDA method. The training pipeline starts with selective pseudo labeling target data using the pre-trained source model. This is followed by fine-tuning a pre-trained text-to-image diffusion model on the target images using textual inversion textual_inversion. Subsequently, the pre-trained source model is used to fine-tune this diffusion model using AlignProp alignprop to generate Source Images. Finally, the finetuned diffusion models are used to generate intermediate domains between source and target domains to perform unsupervised domain adaptation.
  • Figure 2: Visualization of the diffusion-based domain mixup for Unsupervised Domain Adaptation.