Learning to Expand Images for Efficient Visual Autoregressive Modeling

TL;DR

This work tackles the slow iteration of autoregressive image generation by introducing Expanding Autoregressive Modeling (EAR), which leverages spiral center-outward token unfolding and next-any-tokens parallelization to enable efficient, high-fidelity generation. EAR preserves spatial contiguity and enables controlled multi-token predictions per step, aided by a tailor-made causal mask and a KV-cache for fast inference. On ImageNet 256×256, EAR achieves state-of-the-art efficiency–quality trade-offs, with variants reaching low FID and high IS at substantially reduced GFLOPs and steps compared to prior AR methods. The approach also supports flexible image-extension tasks, illustrating the model’s robustness and perceptual alignment with human visual cognition.

Abstract

Autoregressive models have recently shown great promise in visual generation by leveraging discrete token sequences akin to language modeling. However, existing approaches often suffer from inefficiency, either due to token-by-token decoding or the complexity of multi-scale representations. In this work, we introduce Expanding Autoregressive Representation (EAR), a novel generation paradigm that emulates the human visual system's center-outward perception pattern. EAR unfolds image tokens in a spiral order from the center and progressively expands outward, preserving spatial continuity and enabling efficient parallel decoding. To further enhance flexibility and speed, we propose a length-adaptive decoding strategy that dynamically adjusts the number of tokens predicted at each step. This biologically inspired design not only reduces computational cost but also improves generation quality by aligning the generation order with perceptual relevance. Extensive experiments on ImageNet demonstrate that EAR achieves state-of-the-art trade-offs between fidelity and efficiency on single-scale autoregressive models, setting a new direction for scalable and cognitively aligned autoregressive image generation.

Figures (18)

• Figure 1: Scaling behavior of different generation methods on ImageNet $256\times256$ generation benchmark.
• Figure 2: The proposed spiral unfolding and parallel generation strategy. The generation starts from the center of an image, and expands outwards spirally. Each ${S}$ stands for a generation step.
• Figure 3: Illustration of human visual perception. The observations fixate initially at the center of an image, then gradually expand outwards.
• Figure 4: Training EAR transformer on tokens. Note that [S] denotes the start token derived from the class embedding. In practice, the Mask Token also attends to [S] through the attention mechanism to capture class information, although this detail is omitted in the figure for clarity.
• Figure 5: EAR generation process.