Insert Anything: Image Insertion via In-Context Editing in DiT

TL;DR

Insert Anything advances reference-based image editing by unifying mask- and text-guided insertion within a single DiT-based framework. The AnyInsertion dataset enables broad task coverage (person, object, garment) and supports two prompting modalities through polyptych in-context editing (diptych for masks, triptych for text). The approach achieves state-of-the-art results across multiple benchmarks (AnyInsertion, DreamBooth, VTON-HD) and offers practical benefits for creative content, virtual try-on, and scene composition. Ablation studies confirm the value of in-context editing, semantic guidance, and dataset scale for preserving high-frequency details and semantic fidelity.

Abstract

This work presents Insert Anything, a unified framework for reference-based image insertion that seamlessly integrates objects from reference images into target scenes under flexible, user-specified control guidance. Instead of training separate models for individual tasks, our approach is trained once on our new AnyInsertion dataset--comprising 120K prompt-image pairs covering diverse tasks such as person, object, and garment insertion--and effortlessly generalizes to a wide range of insertion scenarios. Such a challenging setting requires capturing both identity features and fine-grained details, while allowing versatile local adaptations in style, color, and texture. To this end, we propose to leverage the multimodal attention of the Diffusion Transformer (DiT) to support both mask- and text-guided editing. Furthermore, we introduce an in-context editing mechanism that treats the reference image as contextual information, employing two prompting strategies to harmonize the inserted elements with the target scene while faithfully preserving their distinctive features. Extensive experiments on AnyInsertion, DreamBooth, and VTON-HD benchmarks demonstrate that our method consistently outperforms existing alternatives, underscoring its great potential in real-world applications such as creative content generation, virtual try-on, and scene composition.

Figures (8)

• Figure 1: Applications of Insert Anything. Our unified image insertion framework supports diverse practical scenarios, including artistic creation, realistic face swapping, cinematic scene composition, virtual garment try-on, accessory customization, and digital prop replacement, demonstrating its versatility and effectiveness in various image editing tasks.
• Figure 2: Overview of AnyInsertion dataset, highlighting its source and example (a) and diversity (b).
• Figure 3: Overview of the Insert Anything model framework. Given different types of prompts, our unified framework processes polyptych inputs (concatenation of reference, source, and masks) through a frozen VAE encoder to preserve high-frequency details, and extracts semantic guidance from image and text encoders. These embeddings are combined and fed into learnable DiT transformer blocks for in-context learning, enabling precise and flexible image insertion guided by either mask- or text-prompt.
• Figure 4: Qualitative comparison of mask-prompt image insertion results. Our method consistently preserves identity and maintains visual coherence across diverse insertion tasks (person, object, and garment) compared to existing methods (AnyDoor chen2024anydoor, MimicBrush chen2024zero, Ace++ mao2025ace++, OOTD xu2024ootdiffusion, CatVTON chong2024catvton).
• Figure 5: Qualitative comparisons with AnyEdit yu2024anyedit on text-prompt object and garment insertion. See Table \ref{['tab:text_comparison_object']} for quantitative results.