Few-Shot Generative Model Adaption via Identity Injection and Preservation

Abstract

Training generative models with limited data presents severe challenges of mode collapse. A common approach is to adapt a large pretrained generative model upon a target domain with very few samples (fewer than 10), known as few-shot generative model adaptation. However, existing methods often suffer from forgetting source domain identity knowledge during adaptation, which degrades the quality of generated images in the target domain. To address this, we propose Identity Injection and Preservation (I$^2$P), which leverages identity injection and consistency alignment to preserve the source identity knowledge. Specifically, we first introduce an identity injection module that integrates source domain identity knowledge into the target domain's latent space, ensuring the generated images retain key identity knowledge of the source domain. Second, we design an identity substitution module, which includes a style-content decoupler and a reconstruction modulator, to further enhance source domain identity preservation. We enforce identity consistency constraints by aligning features from identity substitution, thereby preserving identity knowledge. Both quantitative and qualitative experiments show that our method achieves substantial improvements over state-of-the-art methods on multiple public datasets and 5 metrics.

Figures (12)

• Figure 1: Few-shot generative model. Given a source generative model $G_S$ trained on a large-scale training dataset (such as FFHQ), and adapt to the target domain to get the target generative model $G_T$ by using extremely few (such as 10) training datasets
• Figure 2: The framework of our method. a) Identity injection obtains and fuses the latent features of the mapping networks in the source and target domains, thereby guiding the target domain mapping network to remain the source domain identity knowledge. b) Identity substitution module includes a style-content decoupler that decomposes style information features and identity knowledge features, and a reconstruction modulator for reconstructing features using style information features and identity knowledge features. c) Identity consistency consist of style constraint $\mathcal{L}_s$, content constraint $\mathcal{L}_c$, and synthesis constraint $\mathcal{L}_r$. d) Internal structure of style-content decoupler. e) Internals of the reconstruction modulator.
• Figure 3: Cross-domain adaptation performance comparison on Flickr-Faces-HQ (FFHQ)$\rightarrow$Sketches task. (a) Baseline methods (TGAN, FreezeD, MineGAN) exhibit overfitting artifacts in 10-shot setting; (b) Recent approaches (CDC, RSSA, PIR) show improved alignment but suffer from content distortion or style inconsistency; (c) Our I2P framework maintains structural fidelity and stylistic coherence through identity-preserved adaptation. Visualizations demonstrate our method's capability to generate high-fidelity sketches while preserving source domain identity features from identical latent codes, significantly outperforming state-of-the-art methods in both quality and cross-domain consistency.
• Figure 4: The cross-domain adaptation results of our I2P are shown in the conversion scenarios between FFHQ and four different target domains: FFHQ$\rightarrow$Sketches, FFHQ$\rightarrow$Sunglasses, FFHQ$\rightarrow$Metfaces and FFHQ$\rightarrow$Babies, Qualitative results for each target domain illustrate the ability of our I2P to adapt the same source domain model to different target domains while preserving domain-invariant features.
• Figure 5: The cross-domain adaptation results of our I2P are shown in four different conversion scenarios: (a) LSUN-Churches $\rightarrow$ Haunted-Houses/VanGogh(top left); (b) LSUN-Cars $\rightarrow$ Haunted-Cars/Landscape-Cars(top right); (c) AFHQ-Cat $\rightarrow$ VanGogh(bottom left); (d) AFHQ-Dog $\rightarrow$ Impressionism(bottom right). Each quadrant illustrates our I2P's capability in preserving domain-invariant features while adapting to diverse artistic styles.