Towards Generalized Multi-Image Editing for Unified Multimodal Models

TL;DR

The paper targets the bottleneck of visual inconsistency and poor generalization in Unified Multimodal Models when editing across multiple reference images. It introduces two algorithmic innovations—learnable latent separators and generalized sinusoidal index embeddings—to explicitly distinguish image identities and extrapolate across varying input counts within a hybrid MLLM-Diffusion backbone. To support robust training and evaluation, it also presents inverse-editing-based data construction and the MMIE-Bench, a comprehensive benchmark spanning six editing tasks and 274 examples, assessed by two MLLMs across semantic, visual, and integration dimensions. Empirical results show improved semantic fidelity, visual fidelity, and cross-image integration, with strong generalization to unseen numbers of input images, signaling practical impact for multi-image editing tasks across diverse applications.

Abstract

Unified Multimodal Models (UMMs) integrate multimodal understanding and generation, yet they are limited to maintaining visual consistency and disambiguating visual cues when referencing details across multiple input images. In this work, we propose a scalable multi-image editing framework for UMMs that explicitly distinguishes image identities and generalizes to variable input counts. Algorithmically, we introduce two innovations: 1) The learnable latent separators explicitly differentiate each reference image in the latent space, enabling accurate and disentangled conditioning. 2) The sinusoidal index encoding assigns visual tokens from the same image a continuous sinusoidal index embedding, which provides explicit image identity while allowing generalization and extrapolation on a variable number of inputs. To facilitate training and evaluation, we establish a high-fidelity benchmark using an inverse dataset construction methodology to guarantee artifact-free, achievable outputs. Experiments show clear improvements in semantic consistency, visual fidelity, and cross-image integration over prior baselines on diverse multi-image editing tasks, validating our advantages on consistency and generalization ability.

Figures (13)

• Figure 1: Demonstration of the encoding of visual tokens behind the VAE in hybrid UMM and our design. The arrangement of visual tokens lacks separation and the awareness of the image index. This can lead to confusion of instance identities, misinterpretation of the image index, and a lack of generation of an unseen number of input images.
• Figure 2: Task distribution and editing examples of the MMIE-Bench. The benchmark consists of six different editing tasks involving add, human, replace, style, reasoning, and mixed editing. These tasks also cover different objects, scenarios, and numbers of input images for comprehensive evaluation. All human portraits are from PIE and Echo-4o ju2023directye2025echo.
• Figure 3: Radar evaluation across six multi-image editing tasks by Doubao-1.6. Each radar chart compares four models over the three metrics: Semantic Consistency (SC), Visual Fidelity (VF), and Multi-image Integration (MI). The metric score is rated from 1 to 5.
• Figure 4: Qualitative comparison on representative MMIE-Bench tasks. Our method produces geometrically aligned, instruction-consistent, and compositionally coherent results across addition, replacement, texture transfer, and multi-style fusion tasks. All human data is from Echo-4o and PIE ye2025echoju2023direct.
• Figure 5: Qualitative results for ablation study. Removing the component may cause failure to cross-image reference and editing.