LooseRoPE: Content-aware Attention Manipulation for Semantic Harmonization

TL;DR

LooseRoPE tackles prompt-free image editing by enabling content-aware, semantically harmonious edits through saliency-guided attention modulation. By relaxing RoPE with an inverse range factor $r(S(q))$ and adaptively scaling attention with $k(S(q))$, the method balances preserving the pasted object's identity with contextual harmonization; a Vision-Language Model guides parameter steering during inference. Built atop FLUX Kontext, the approach delivers seamless crop-and-paste results without textual prompts, validated on a 150-sample benchmark with both quantitative metrics (CLIP-IQA, LPIPS) and a user study demonstrating improved identity preservation and blending. The work highlights attention as a controllable, content-aware mechanism for image editing and outlines directions for future work, including video extension and deeper analysis of model attention dynamics.

Abstract

Recent diffusion-based image editing methods commonly rely on text or high-level instructions to guide the generation process, offering intuitive but coarse control. In contrast, we focus on explicit, prompt-free editing, where the user directly specifies the modification by cropping and pasting an object or sub-object into a chosen location within an image. This operation affords precise spatial and visual control, yet it introduces a fundamental challenge: preserving the identity of the pasted object while harmonizing it with its new context. We observe that attention maps in diffusion-based editing models inherently govern whether image regions are preserved or adapted for coherence. Building on this insight, we introduce LooseRoPE, a saliency-guided modulation of rotational positional encoding (RoPE) that loosens the positional constraints to continuously control the attention field of view. By relaxing RoPE in this manner, our method smoothly steers the model's focus between faithful preservation of the input image and coherent harmonization of the inserted object, enabling a balanced trade-off between identity retention and contextual blending. Our approach provides a flexible and intuitive framework for image editing, achieving seamless compositional results without textual descriptions or complex user input.

Figures (16)

• Figure 1: We introduce LooseRoPE, a training-free image editing algorithm that turns crudely edited inputs (top row) into coherent, high-quality results (bottom row). In each example, cropped regions are pasted either from other images (blue frames) or moved within the same image (magenta frames), sometimes leaving holes behind. Without any text prompts or additional supervision, LooseRoPE harmonizes the pasted content with its new context, producing seamless and semantically consistent outputs.
• Figure 2: Examples of Neglect and Suppression failure modes in vanilla FLUX Kontext. In all the shown examples, we instruct the model with: "blend the cropped objects into the image in a convincing manner."
• Figure 3: Saliency-Guided Attention Manipulation. Given an image with a crudely pasted crop, we smoothly blend it into the surrounding scene by manipulating the attention computation during inference using a saliency map of the cropped region. Output-image queries (within the dotted blue frame) attend to input-image keys using RoPE with a saliency-dependent range factor $r(S(q))$, which scales the positional coordinate and controls the spread of attention ("Rotated"). The corresponding attention logits in the crop mask are then scaled by $k(S(q))$ ("Scaled"). High-saliency queries (red) have $r(S(q))\!\approx\!1$ and $k(S(q))\!>\!1$, keeping attention localized and preserving identity, evident in the gorilla's facial expression. Low-saliency queries (blue) have smaller $r(S(q))$ and $k(S(q))\!<\!1$, broadening attention and reducing crop-internal focus. This enables semantic blending with surrounding context, as seen in the forehead query attending to the hood and integrating smoothly in the final result. The "Default" attention map is shown for reference only and is not used in our method.
• Figure 4: Attention Map Visualization . Top: For a query on the bike wheel, vanilla Kontext (b) produces highly local attention, whereas our method (c) correctly attends to the gear wheel, enabling coherent blending (e). Bottom: For a query on the duck's neck, Kontext (b) again attends locally within the pasted crop. In contrast, our RoPE modification (c) captures the semantic relation to the giraffe's neck, resulting in a seamless blend (e).
• Figure 5: VLM guided manipulation of attention. Even inputs that exhibit severe neglect or suppression are eventually edited successfully. Green arrows indicate a downscale in the saliency map (neglect), and Orange arrows indicate an upscale (suppression). The figure shows the input, followed by three $\hat{x}_0$ predictions at timestep 2, and our method's final output.