Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Turn Toward Better Alignment: Few-Shot Generative Adaptation with Equivariant Feature Rotation

Chenghao Xu, Qi Liu, Jiexi Yan, Muli Yang, Cheng Deng

TL;DR

Equivariant Feature Rotation (EFR), a novel adaptation strategy that aligns source and target domains at two complementary levels within a self-rotated proxy feature space, is proposed and significantly enhances the generative performance within the targeted domain.

Abstract

Few-shot image generation aims to effectively adapt a source generative model to a target domain using very few training images. Most existing approaches introduce consistency constraints-typically through instance-level or distribution-level loss functions-to directly align the distribution patterns of source and target domains within their respective latent spaces. However, these strategies often fall short: overly strict constraints can amplify the negative effects of the domain gap, leading to distorted or uninformative content, while overly relaxed constraints may fail to leverage the source domain effectively. This limitation primarily stems from the inherent discrepancy in the underlying distribution structures of the source and target domains. The scarcity of target samples further compounds this issue by hindering accurate estimation of the target domain's distribution. To overcome these limitations, we propose Equivariant Feature Rotation (EFR), a novel adaptation strategy that aligns source and target domains at two complementary levels within a self-rotated proxy feature space. Specifically, we perform adaptive rotations within a parameterized Lie Group to transform both source and target features into an equivariant proxy space, where alignment is conducted. These learnable rotation matrices serve to bridge the domain gap by preserving intra-domain structural information without distortion, while the alignment optimization facilitates effective knowledge transfer from the source to the target domain. Comprehensive experiments on a variety of commonly used datasets demonstrate that our method significantly enhances the generative performance within the targeted domain.

A Turn Toward Better Alignment: Few-Shot Generative Adaptation with Equivariant Feature Rotation

TL;DR

Equivariant Feature Rotation (EFR), a novel adaptation strategy that aligns source and target domains at two complementary levels within a self-rotated proxy feature space, is proposed and significantly enhances the generative performance within the targeted domain.

Abstract

Few-shot image generation aims to effectively adapt a source generative model to a target domain using very few training images. Most existing approaches introduce consistency constraints-typically through instance-level or distribution-level loss functions-to directly align the distribution patterns of source and target domains within their respective latent spaces. However, these strategies often fall short: overly strict constraints can amplify the negative effects of the domain gap, leading to distorted or uninformative content, while overly relaxed constraints may fail to leverage the source domain effectively. This limitation primarily stems from the inherent discrepancy in the underlying distribution structures of the source and target domains. The scarcity of target samples further compounds this issue by hindering accurate estimation of the target domain's distribution. To overcome these limitations, we propose Equivariant Feature Rotation (EFR), a novel adaptation strategy that aligns source and target domains at two complementary levels within a self-rotated proxy feature space. Specifically, we perform adaptive rotations within a parameterized Lie Group to transform both source and target features into an equivariant proxy space, where alignment is conducted. These learnable rotation matrices serve to bridge the domain gap by preserving intra-domain structural information without distortion, while the alignment optimization facilitates effective knowledge transfer from the source to the target domain. Comprehensive experiments on a variety of commonly used datasets demonstrate that our method significantly enhances the generative performance within the targeted domain.
Paper Structure (18 sections, 9 equations, 7 figures, 3 tables)

This paper contains 18 sections, 9 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Image Generation with Limited Training Data.
  4. Few-shot Generative Model Adaptation.
  5. Method
  6. Preliminaries
  7. Problem formulation.
  8. Main Challenge and Contribution.
  9. Equivariant Self-Rotated Proxy Feature Space Projection
  10. Optimization in Equivariant Proxy Feature Space
  11. Instance-wise Alignment.
  12. Distribution-level Matching.
  13. Overall.
  14. Experiments
  15. Experimental Settings
  16. ...and 3 more sections

Figures (7)

  • Figure 1: Comparison between the existing naive direct alignment and our self-rotated alignment.
  • Figure 2: The overall framework of our proposed method.
  • Figure 3: Generative adaptation results of our method on Cars$\rightarrow$Wrecked cars and Church$\rightarrow$Haunted house (10-shot). Best zoomed in and viewed in color.
  • Figure 4: Visualized comparison results with different methods on FFHQ$\rightarrow$Sketches and FFHQ$\rightarrow$AFHQ-Cat adaptations. The target training data is under the 10-shot setting. Synthesized samples in each column are generated with the same random input $\mathbf{z}$. Best zoomed in and viewed in color. Best zoomed in and viewed in color.
  • Figure 5: Latent space interpolation results on FFHQ$\rightarrow$VanGoGh and FFHQ$\rightarrow$Amedeo
  • ...and 2 more figures