Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SF(DA)$^2$: Source-free Domain Adaptation Through the Lens of Data Augmentation

Uiwon Hwang, Jonghyun Lee, Juhyeon Shin, Sungroh Yoon

TL;DR

This paper constructs an augmentation graph in the feature space of the pretrained model using the neighbor relationships between target features and proposes spectral neighborhood clustering to identify partitions in the prediction space and proposes implicit feature augmentation and feature disentanglement as regularization loss functions that effectively utilize class semantic information within the feature space.

Abstract

In the face of the deep learning model's vulnerability to domain shift, source-free domain adaptation (SFDA) methods have been proposed to adapt models to new, unseen target domains without requiring access to source domain data. Although the potential benefits of applying data augmentation to SFDA are attractive, several challenges arise such as the dependence on prior knowledge of class-preserving transformations and the increase in memory and computational requirements. In this paper, we propose Source-free Domain Adaptation Through the Lens of Data Augmentation (SF(DA)$^2$), a novel approach that leverages the benefits of data augmentation without suffering from these challenges. We construct an augmentation graph in the feature space of the pretrained model using the neighbor relationships between target features and propose spectral neighborhood clustering to identify partitions in the prediction space. Furthermore, we propose implicit feature augmentation and feature disentanglement as regularization loss functions that effectively utilize class semantic information within the feature space. These regularizers simulate the inclusion of an unlimited number of augmented target features into the augmentation graph while minimizing computational and memory demands. Our method shows superior adaptation performance in SFDA scenarios, including 2D image and 3D point cloud datasets and a highly imbalanced dataset.

SF(DA)$^2$: Source-free Domain Adaptation Through the Lens of Data Augmentation

TL;DR

This paper constructs an augmentation graph in the feature space of the pretrained model using the neighbor relationships between target features and proposes spectral neighborhood clustering to identify partitions in the prediction space and proposes implicit feature augmentation and feature disentanglement as regularization loss functions that effectively utilize class semantic information within the feature space.

Abstract

In the face of the deep learning model's vulnerability to domain shift, source-free domain adaptation (SFDA) methods have been proposed to adapt models to new, unseen target domains without requiring access to source domain data. Although the potential benefits of applying data augmentation to SFDA are attractive, several challenges arise such as the dependence on prior knowledge of class-preserving transformations and the increase in memory and computational requirements. In this paper, we propose Source-free Domain Adaptation Through the Lens of Data Augmentation (SF(DA)), a novel approach that leverages the benefits of data augmentation without suffering from these challenges. We construct an augmentation graph in the feature space of the pretrained model using the neighbor relationships between target features and propose spectral neighborhood clustering to identify partitions in the prediction space. Furthermore, we propose implicit feature augmentation and feature disentanglement as regularization loss functions that effectively utilize class semantic information within the feature space. These regularizers simulate the inclusion of an unlimited number of augmented target features into the augmentation graph while minimizing computational and memory demands. Our method shows superior adaptation performance in SFDA scenarios, including 2D image and 3D point cloud datasets and a highly imbalanced dataset.
Paper Structure (34 sections, 4 theorems, 14 equations, 7 figures, 13 tables, 1 algorithm)

This paper contains 34 sections, 4 theorems, 14 equations, 7 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Source-free Domain Adaptation
  4. Data Augmentation
  5. Spectral Contrastive Learning
  6. Proposed Method
  7. Source-free Domain Adaptation Scenario
  8. Augmentation Graph on Feature Space
  9. Finding Partition on Prediction Space
  10. Implicit Feature Augmentation
  11. Experiments
  12. Implementation details
  13. Evaluation Results
  14. 2D and 3D Datasets
  15. Imbalanced Dataset
  16. ...and 19 more sections

Key Result

Lemma 1

Let $L$ be the normalized Laplacian matrix of the augmentation graph $G(\mathcal{X},l)$. The matrix $H$, which has the eigenvectors corresponding to the $k$ largest eigenvalues of $L$ as its columns, can be learned as a function $h$ by minimizing the following matrix factorization loss: where $x_i$ and $x_j$ are vertices of the augmentation graph, $l_{x_ix_j}$ is an edge between $x_i$ and $x_j$,

Figures (7)

  • Figure 1: Overview of SF(DA)$^2$. Here, $f$ and $g$ indicate the feature extractor and the classifier, $\mathbf{z}$ denotes a target feature, $\Sigma$ is a covariance matrix for a class, and $dist(\cdot,\cdot)$ denotes a distance measure.
  • Figure 2: Effectiveness of implicit feature augmentation.
  • Figure 3: Ablation study on loss functions for implicit feature augmentation.
  • Figure 4: tSNE visualization of the feature space before and after adaptation.
  • Figure 5: Ablation study on hyperparameters for spectral neighbor clustering.
  • ...and 2 more figures

Theorems & Definitions (5)

  • Lemma 1: spectralcl
  • Proposition 1
  • Proposition 2
  • proof
  • Lemma 2