Non-Markov Multi-Round Conversational Image Generation with History-Conditioned MLLMs

TL;DR

This work formalizes non-Markov, non-linear multi-round conversational image generation, where later instructions depend on long-range history rather than only the most recent output. It introduces rollback-style editing and name-based personalization datasets to enforce retrieval of earlier states and long-range identity bindings, paired with a history-conditioned training framework and token-level caching to mitigate drift. Two enabling components—reconstruction-based DiT detokenization and a multi-stage instruction fine-tuning curriculum—substantially improve fidelity and editable personalization without sacrificing single-turn performance. The results demonstrate improved non-Markov consistency and instruction compliance, with practical implications for more faithful, interactive visual systems that maintain identity and continuity across long conversations.

Abstract

Conversational image generation requires a model to follow user instructions across multiple rounds of interaction, grounded in interleaved text and images that accumulate as chat history. While recent multimodal large language models (MLLMs) can generate and edit images, most existing multi-turn benchmarks and training recipes are effectively Markov: the next output depends primarily on the most recent image, enabling shortcut solutions that ignore long-range history. In this work we formalize and target the more challenging non-Markov setting, where a user may refer back to earlier states, undo changes, or reference entities introduced several rounds ago. We present (i) non-Markov multi-round data construction strategies, including rollback-style editing that forces retrieval of earlier visual states and name-based multi-round personalization that binds names to appearances across rounds; (ii) a history-conditioned training and inference framework with token-level caching to prevent multi-round identity drift; and (iii) enabling improvements for high-fidelity image reconstruction and editable personalization, including a reconstruction-based DiT detokenizer and a multi-stage fine-tuning curriculum. We demonstrate that explicitly training for non-Markov interactions yields substantial improvements in multi-round consistency and instruction compliance, while maintaining strong single-round editing and personalization.

Figures (19)

• Figure 1: Non-Markov vs. Markov multi-round generation. Markov: each turn depends primarily on the latest image. Non-Markov: later turns refer to earlier states (rollback/undo) or to entities introduced multiple rounds ago (name-based references).
• Figure 2: Rollback-style non-Markov editing construction. Starting from Markov editing chains, we add alternative edits from earlier states and synthesize final-round rollback instructions so the correct target depends on an earlier image rather than the latest one.
• Figure 3: Name-based non-Markov multi-round personalization from video. Round 1 introduces and names multiple people. Later rounds request portraits by name only. Supervision comes from video-derived identity targets, enforcing persistent name$\leftrightarrow$appearance binding across rounds.
• Figure 4: Overview of our framework and data pipeline.
• Figure 5: Illustration of accumulated error when encoding and decoding an image several times. This result suggests to caching image tokens in chat history instead of image pixels when performing multi-round inference.