InfiniteYou: Flexible Photo Recrafting While Preserving Your Identity

TL;DR

InfiniteYou (InfU) tackles identity-preserved image recrafting by leveraging Diffusion Transformers with a novel InfuseNet that injects identity features via residual connections, leaving the base model's generation ability largely intact. A multi-stage training regime, including synthetic SPMS data, enhances text-image alignment and aesthetics, achieving state-of-the-art identity similarity and image quality. The method is plug-and-play, compatible with existing DiT variants and adapters, and ablation studies confirm the benefits of residual identity injection and SPMS-driven fine-tuning. Together, these contributions advance practical, high-fidelity personalized image generation while maintaining model flexibility for broader adoption.

Abstract

Achieving flexible and high-fidelity identity-preserved image generation remains formidable, particularly with advanced Diffusion Transformers (DiTs) like FLUX. We introduce InfiniteYou (InfU), one of the earliest robust frameworks leveraging DiTs for this task. InfU addresses significant issues of existing methods, such as insufficient identity similarity, poor text-image alignment, and low generation quality and aesthetics. Central to InfU is InfuseNet, a component that injects identity features into the DiT base model via residual connections, enhancing identity similarity while maintaining generation capabilities. A multi-stage training strategy, including pretraining and supervised fine-tuning (SFT) with synthetic single-person-multiple-sample (SPMS) data, further improves text-image alignment, ameliorates image quality, and alleviates face copy-pasting. Extensive experiments demonstrate that InfU achieves state-of-the-art performance, surpassing existing baselines. In addition, the plug-and-play design of InfU ensures compatibility with various existing methods, offering a valuable contribution to the broader community.

Figures (6)

• Figure 1: InfiniteYou generates identity-preserved images with exceptional identity similarity, text-image alignment, quality, and aesthetics.
• Figure 2: The superiority of the DiT-based method over the U-Net-based one and the side effects of IP-Adapter (IPA) ipa.
• Figure 3: The main framework of InfiniteYou (InfU) and the detailed architecture of InfuseNet. The projected identity features and an optional control image are injected by InfuseNet into text-to-image DiTs via residual connections. Specifically, each DiT block in InfuseNet predicts the output residuals of the corresponding $i$ DiT blocks in the base model. Only InfuseNet and the projection network are trainable.
• Figure 4: The introduced multi-stage training strategy with synthetic single-person-multiple-sample (SPMS) data and supervised fine-tuning (SFT).
• Figure 5: Qualitative comparison results of InfU with the state-of-the-art baselines, FLUX.1-dev IP-Adapter instantxfluxipa and PuLID-FLUX pulid.