PrimeComposer: Faster Progressively Combined Diffusion for Image Composition with Attention Steering

TL;DR

PrimeComposer reframes image composition as subject-guided local foreground editing and introduces a fast, training-free diffusion framework. It leverages the Correlation Diffuser to extract and infuse prior attention weights into the Latent Diffusion Model and employs Region-constrained Cross-Attention to localize object-token influence, complemented by an extended classifier-free guidance strategy. The approach achieves superior cross-domain synthesis quality and faster inference on the tf-icon benchmark, with robust ablations highlighting the contributions of RCA and attention steering. This work enables efficient, high-fidelity foreground insertion across domains, reducing artifacts in transition areas and improving scene coherence without retraining pre-trained models.

Abstract

Image composition involves seamlessly integrating given objects into a specific visual context. Current training-free methods rely on composing attention weights from several samplers to guide the generator. However, since these weights are derived from disparate contexts, their combination leads to coherence confusion and loss of appearance information. These issues worsen with their excessive focus on background generation, even when unnecessary in this task. This not only impedes their swift implementation but also compromises foreground generation quality. Moreover, these methods introduce unwanted artifacts in the transition area. In this paper, we formulate image composition as a subject-based local editing task, solely focusing on foreground generation. At each step, the edited foreground is combined with the noisy background to maintain scene consistency. To address the remaining issues, we propose PrimeComposer, a faster training-free diffuser that composites the images by well-designed attention steering across different noise levels. This steering is predominantly achieved by our Correlation Diffuser, utilizing its self-attention layers at each step. Within these layers, the synthesized subject interacts with both the referenced object and background, capturing intricate details and coherent relationships. This prior information is encoded into the attention weights, which are then integrated into the self-attention layers of the generator to guide the synthesis process. Besides, we introduce a Region-constrained Cross-Attention to confine the impact of specific subject-related tokens to desired regions, addressing the unwanted artifacts shown in the prior method thereby further improving the coherence in the transition area. Our method exhibits the fastest inference efficiency and extensive experiments demonstrate our superiority both qualitatively and quantitatively.

Figures (11)

• Figure 1: Current methods encounter significant challenges in preserving the objects’ appearance (left) and synthesizing natural coherence (right). The problematic areas of coherence are indicated by red dotted lines.
• Figure 2: The overview of our PrimeComposer.
• Figure 3: The effectiveness of our Region-constrained Cross Attention.
• Figure 4: Qualitative results regarding the unexpected coherence problem, i.e., style inconsistency.
• Figure 5: Qualitative comparison with SOTA baselines in cross-domain image composition. All the results of TF-ICON come from its original paper.