NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generation

TL;DR

NextFlow introduces a unified decoder-only Transformer trained on $6$ trillion interleaved text-image tokens that activates both multimodal understanding and generation via next-scale prediction for visuals and next-token prediction for text. It pairs a dual-codebook tokenizer with a shared output head and Multiscale 3D RoPE, optionally complemented by a diffusion decoder, to achieve fast $1024\times1024$ image synthesis and robust image editing within a single architecture. The paper details a robust training regimen (scale-aware loss, self-correction, prefix-tuning GRPO RL, and progressive resolution pretraining) and demonstrates state-of-the-art or competitive results across image generation, editing, and interleaved tasks, along with strong efficiency gains. Empirically, NextFlow matches or surpasses specialized diffusion baselines in visual quality while preserving LLM-like reasoning and in-context learning, signaling a practical path toward unified multimodal models with real-time interactive capabilities.

Abstract

We present NextFlow, a unified decoder-only autoregressive transformer trained on 6 trillion interleaved text-image discrete tokens. By leveraging a unified vision representation within a unified autoregressive architecture, NextFlow natively activates multimodal understanding and generation capabilities, unlocking abilities of image editing, interleaved content and video generation. Motivated by the distinct nature of modalities - where text is strictly sequential and images are inherently hierarchical - we retain next-token prediction for text but adopt next-scale prediction for visual generation. This departs from traditional raster-scan methods, enabling the generation of 1024x1024 images in just 5 seconds - orders of magnitude faster than comparable AR models. We address the instabilities of multi-scale generation through a robust training recipe. Furthermore, we introduce a prefix-tuning strategy for reinforcement learning. Experiments demonstrate that NextFlow achieves state-of-the-art performance among unified models and rivals specialized diffusion baselines in visual quality.

Figures (25)

• Figure 1: Architecture of NextFlow. NextFlow processes interleaved text-image discrete token sequences as input and generates interleaved multimodal outputs. Text tokens are predicted via next-token modeling, while visual tokens are generated through next-scale prediction.
• Figure 2: NextFlow visualization. Our approach generates high-fidelity images via a pure discrete autoregressive framework, achieving production-grade visual quality.
• Figure 3: NextFlow visualization. Utilizing next-scale prediction for visual generation, our model can efficiently synthesizes high-quality $1024 \times 1024$ images under 5 seconds.
• Figure 4: Edit results of NextFlow on EditCanvas benchmark.
• Figure 5: Multi-Scale 3D RoPE for interleaved text-image sequences. Text tokens employ diagonal positions (e.g., position 3 → [3,3,3]), while vision tokens utilize normalized spatial coordinates with augmented scale indices. For clarity, we show 2 scales using square images as examples. Note that the next-scale prediction paradigm excludes 1×1 features in the input, beginning with 2×2 feature maps.