AnyMS: Bottom-up Attention Decoupling for Layout-guided and Training-free Multi-subject Customization

TL;DR

AnyMS tackles training-free layout-guided multi-subject image synthesis by decoupling textual, visual, and layout conditions through a global cross-attention separation and a local, layout-constrained attention mechanism, augmented by pre-trained image adapters. The approach yields a dual-level attention decoupling: global decoupling preserves text alignment while preventing text-image conflicts, and local decoupling confines each subject to its designated layout region to maintain identity and layout fidelity. It eliminates the need for subject-specific learning or adapter tuning, enabling scalable generation of complex scenes with many subjects. Empirical results on diverse benchmarks show state-of-the-art performance in layout control, text alignment, and identity preservation, with efficient inference and strong generalization to larger subject counts. These advances offer practical impact for personalized content creation in media and advertising, and pave the way for extensions to video and joint subject-action/style customization.

Abstract

Multi-subject customization aims to synthesize multiple user-specified subjects into a coherent image. To address issues such as subjects missing or conflicts, recent works incorporate layout guidance to provide explicit spatial constraints. However, existing methods still struggle to balance three critical objectives: text alignment, subject identity preservation, and layout control, while the reliance on additional training further limits their scalability and efficiency. In this paper, we present AnyMS, a novel training-free framework for layout-guided multi-subject customization. AnyMS leverages three input conditions: text prompt, subject images, and layout constraints, and introduces a bottom-up dual-level attention decoupling mechanism to harmonize their integration during generation. Specifically, global decoupling separates cross-attention between textual and visual conditions to ensure text alignment. Local decoupling confines each subject's attention to its designated area, which prevents subject conflicts and thus guarantees identity preservation and layout control. Moreover, AnyMS employs pre-trained image adapters to extract subject-specific features aligned with the diffusion model, removing the need for subject learning or adapter tuning. Extensive experiments demonstrate that AnyMS achieves state-of-the-art performance, supporting complex compositions and scaling to a larger number of subjects.

Figures (9)

• Figure 1: AnyMS enables training-free layout-guided multi-subject customization, supporting diverse subject combinations and scaling to larger numbers while maintaining a balance among layout control, text alignment, and identity preservation. See more visualization details and layout configurations in the Appendix.
• Figure 2: Layout-guided Multi-subject Customization Results. (a) Result of latent injection method MuDI jang2024identity. (b) Result of attention rectifying method Cones2 liu2023customizable. (c) Result of adapter tuning method MS-Diffusion wang2024ms. Different colors show the associations between subjects and their corresponding layout configurations.
• Figure 3: The Overview of Pipeline. AnyMS applies a dual-level attention decoupling strategy alongside the general denoising process of the diffusion model. (a) The global decoupling separates cross-attention between text and subject images. (b) The local decoupling further disentangles image cross-attention based on layout constraints. The final $z_0$ is then decoded back to target image $I^G$.
• Figure 4: Detailed Quantitative Results with Different Numbers of Subjects. Marker shape indicates the number of subjects, and color represents the method used. Rhombus denotes aggregated results.
• Figure 5: Comparison of Layout-guided Multi-subject Customization. Different colors show the associations between subjects and their corresponding layout configurations.