FlexIP: Dynamic Control of Preservation and Personality for Customized Image Generation

TL;DR

FlexIP is introduced, a novel framework that decouples identity preservation and personalized manipulation through two dedicated components: a Personalization Adapter for stylistic manipulation and a Preservation Adapter for identity maintenance.

Abstract

With the rapid advancement of 2D generative models, preserving subject identity while enabling diverse editing has emerged as a critical research focus. Existing methods typically face inherent trade-offs between identity preservation and personalized manipulation. We introduce FlexIP, a novel framework that decouples these objectives through two dedicated components: a Personalization Adapter for stylistic manipulation and a Preservation Adapter for identity maintenance. By explicitly injecting both control mechanisms into the generative model, our framework enables flexible parameterized control during inference through dynamic tuning of the weight adapter. Experimental results demonstrate that our approach breaks through the performance limitations of conventional methods, achieving superior identity preservation while supporting more diverse personalized generation capabilities (Project Page: https://flexip-tech.github.io/flexip/).

Figures (10)

• Figure 1: Top: FlexIP showcases versatility and precision in personalized image generation. Given a single reference image (left column), it vividly captures identity details while creatively following diverse text prompts, resulting in coherent yet highly varied edits. Bottom: FlexIP's dynamic weight gating mechanism smoothly transitions between strong identity preservation and diverse personalization, significantly outperforming IP-Adapter, which suffers from abrupt identity shifts and rigid control. This reflects superior flexibility and user-friendly controllability.
• Figure 2: Comparison with other methods on two indicators, image preservation and text fidelity, demonstrates that our approach surpasses previous methods in both aspects.
• Figure 3: The overall pipeline of FlexIP. It introduces three key improvements to the model: the preservation adapter, the personalization adapter, and dynamic weight gating. First, the preservation adapter combines high-level and low-level features to ensure preservation. The personalization adapter interacts with text and visual CLS tokens to absorb meaningful visual cues, grounding textual modifications within a coherent visual context. Finally, dynamic weight gating navigates the trade-off between personalization and preservation more effectively through independent adapters controlled by a dynamic weight gating mechanism.
• Figure 4: Qualitative comparison with other methods. Our approach surpasses alternative methods in its exceptional ability to preserve identity while generating a wide range of diverse and personalized outputs.
• Figure 5: The effectiveness of the dynamic weight gating mechanism.