Few-shot Image Generation via Information Transfer from the Built Geodesic Surface

TL;DR

This work tackles extreme few-shot image generation by proposing ITBGS, a framework built on two modules: FAGS, which constructs a pseudo-source domain by forming a Geodesic surface in the Pre-Shape Space from very few samples, and I&R, which supervises interpolation and regularizes distances to reduce artifacts. FAGS creates a pseudo-source domain from discriminator features projected into the Pre-Shape Space, enforcing Geodesic self-correlation with real features to transfer structural information without a pre-trained source generator. I&R mitigates blur and stair-like interpolation through an interpolated adversarial loss $L_{inp}$ and a KL-divergence-based distance regularizer $L_{dr}$, yielding smoother latent-space traversals. Experiments across diverse 10-shot datasets demonstrate that ITBGS achieves state-of-the-art or competitive fidelity and diversity, validating the method’s data-efficient transfer capability and potential utility for downstream tasks such as classification and segmentation.

Abstract

Images generated by most of generative models trained with limited data often exhibit deficiencies in either fidelity, diversity, or both. One effective solution to address the limitation is few-shot generative model adaption. However, the type of approaches typically rely on a large-scale pre-trained model, serving as a source domain, to facilitate information transfer to the target domain. In this paper, we propose a method called Information Transfer from the Built Geodesic Surface (ITBGS), which contains two module: Feature Augmentation on Geodesic Surface (FAGS); Interpolation and Regularization (I\&R). With the FAGS module, a pseudo-source domain is created by projecting image features from the training dataset into the Pre-Shape Space, subsequently generating new features on the Geodesic surface. Thus, no pre-trained models is needed for the adaption process during the training of generative models with FAGS. I\&R module are introduced for supervising the interpolated images and regularizing their relative distances, respectively, to further enhance the quality of generated images. Through qualitative and quantitative experiments, we demonstrate that the proposed method consistently achieves optimal or comparable results across a diverse range of semantically distinct datasets, even in extremely few-shot scenarios.

Figures (12)

• Figure 1: Our motivation on Information Transfer from the Built Geodesic Surface (ITBGS). A pseudo-source domain is created by manifold data augmenting the features extracted only from extremely few training samples, e.g., 10 samples, and adapt to the target domain for training generator in the Pre-Shape Space. We interpolate the latents within the target domain, ensuring that the generated features maintains a similar spatial distribution to the augmented features. The adaption method is achieved by aligning the inherent structural information of the two aforementioned features. Additionally, the interpolation and regularization strategies are employed to the generated samples and features. The trained generator can be used for further applications, such as few-shot image classification and instance segmentation.
• Figure 2: Illustration of the Feature Augmentation on Geodesic Surface (FAGS) module. We sample coefficients $\omega$ from the Dirichlet distribution and generate an anchor latent $\bar{z}$. Subsequently, we project the feature $\mathcal{D}(\mathcal{G}(\bar{z}))$ into the Pre-Shape Space. Similarly, we project the features extracted from the training set $\mathbb{D}_{real}$ denoted as $\mathcal{D}(x_i)$ and obtain new features $\hat{x}$ from the built Geodesic surface using the same weights $\omega$. Then, we ensure the self-correlation consistency between these two features, $\mathcal{D}(\mathcal{G}(\bar{z}))$ and $\hat{x}$.
• Figure 3: Interpolated images generated by StyleGAN2 with FAGS. Blurriness occurs in the intermediate interpolations.
• Figure 4: Illustration of the I&R module, containing two parts: Interpolation and Regularization.
• Figure 5: Training and generated samples of several methods on Amedeo Modigliani paintings (left) and Landscape drawings (right). Our method exhibits more fidelity and diversity.