Unveil Inversion and Invariance in Flow Transformer for Versatile Image Editing

TL;DR

The paper tackles tuning-free image editing in flow-transformer models by addressing two core challenges: faithful inversion of real images into the model latent space and flexible invariance control to preserve non-target content. It shows that Euler-based inversion in rectified flow incurs larger approximation errors than DDIM, and introduces a two-stage inversion with Stage I fixed-point velocity estimation and Stage II per-step velocity compensation to stay close to the authentic generation process, enhancing editability. For invariance, it proposes AdaLN-based invariance control that replaces unedited text features within AdaLN, linking edits to image semantics and enabling diverse edits from rigid to non-rigid. Empirical results on the PIE dataset demonstrate versatile, high-fidelity editing and robust content preservation, underscoring the potential of flow transformer-based tuning-free editing for real-world applications.

Abstract

Leveraging the large generative prior of the flow transformer for tuning-free image editing requires authentic inversion to project the image into the model's domain and a flexible invariance control mechanism to preserve non-target contents. However, the prevailing diffusion inversion performs deficiently in flow-based models, and the invariance control cannot reconcile diverse rigid and non-rigid editing tasks. To address these, we systematically analyze the \textbf{inversion and invariance} control based on the flow transformer. Specifically, we unveil that the Euler inversion shares a similar structure to DDIM yet is more susceptible to the approximation error. Thus, we propose a two-stage inversion to first refine the velocity estimation and then compensate for the leftover error, which pivots closely to the model prior and benefits editing. Meanwhile, we propose the invariance control that manipulates the text features within the adaptive layer normalization, connecting the changes in the text prompt to image semantics. This mechanism can simultaneously preserve the non-target contents while allowing rigid and non-rigid manipulation, enabling a wide range of editing types such as visual text, quantity, facial expression, etc. Experiments on versatile scenarios validate that our framework achieves flexible and accurate editing, unlocking the potential of the flow transformer for versatile image editing.

Figures (15)

• Figure 1: Our framework reconciles the invariance control for rigid and non-rigid editing, enabling versatile editing via flow transformer.
• Figure 2: Framework of rectified flow inversion and editing. Left: The two-stage inversion for rectified flow. A basic inversion trajectory is first constructed to resemble the generation process, and then mild compensation is added to the velocity to recover the image exactly. Right: Invariance control with text feature replacement within AdaLN during sampling. In the target branch, the unchanged text features are replaced with features from the original image while the edited text features remain intact.
• Figure 3: Comparison of Euler inversion for rectified flow and DDIM inversion. With the same inversion steps as 50, the Euler is evaluated on SD3.5 while the DDIM is evaluated on SD1.5.
• Figure 4: Visualization of latents $\mathbf{x}_t$ w / w.o averaging.
• Figure 5: Invariance control of AdaLN in different editing scenarios. We show the proposed module can adapt to versatile editing types with high fidelity, including non-rigid editing such as altering quantity, facial expression, pose, visual text, etc.