Generation then Reconstruction: Accelerating Masked Autoregressive Models via Two-Stage Sampling

TL;DR

This work tackles the efficiency bottleneck of Masked Autoregressive (MAR) visual generation by introducing Generation then Reconstruction (GtR), a training-free hierarchical sampling strategy. GtR splits generation into a slower stage that establishes global semantic structure and a fast reconstruction stage that completes detail tokens, augmented by Frequency-Weighted Token Selection (FTS) which allocates more diffusion steps to high-frequency, detail-rich tokens. The approach achieves a substantial $3.72\times$ speedup on MAR-H with comparable FID and IS on ImageNet and also improves acceleration in text-to-image generation, outperforming existing MAR acceleration methods. By remaining training-free and compatible with various MAR models, GtR offers a practical, broadly applicable route to efficient parallel visual generation with preserved quality.

Abstract

Masked Autoregressive (MAR) models promise better efficiency in visual generation than autoregressive (AR) models for the ability of parallel generation, yet their acceleration potential remains constrained by the modeling complexity of spatially correlated visual tokens in a single step. To address this limitation, we introduce Generation then Reconstruction (GtR), a training-free hierarchical sampling strategy that decomposes generation into two stages: structure generation establishing global semantic scaffolding, followed by detail reconstruction efficiently completing remaining tokens. Assuming that it is more difficult to create an image from scratch than to complement images based on a basic image framework, GtR is designed to achieve acceleration by computing the reconstruction stage quickly while maintaining the generation quality by computing the generation stage slowly. Moreover, observing that tokens on the details of an image often carry more semantic information than tokens in the salient regions, we further propose Frequency-Weighted Token Selection (FTS) to offer more computation budget to tokens on image details, which are localized based on the energy of high frequency information. Extensive experiments on ImageNet class-conditional and text-to-image generation demonstrate 3.72x speedup on MAR-H while maintaining comparable quality (e.g., FID: 1.59, IS: 304.4 vs. original 1.59, 299.1), substantially outperforming existing acceleration methods across various model scales and generation tasks. Our codes will be released in https://github.com/feihongyan1/GtR.

Figures (6)

• Figure 1: The difference of token features in the adjacent steps. Once a token is decoded, its adjacent tokens tend to exhibit significant changes.
• Figure 2: Correspondence between high-frequency regions in pixel space and feature space. Each triple shows three images: original image, pixel-space high-frequency heatmap via high-pass filtering, and frequency heatmap of MAR's conditioning vectors. The spatial alignment demonstrates that high-frequency tokens in feature space indicate regions with fine-grained textures and high-frequency details.
• Figure 3: Comparison of generation consistency in two sampling methods when 50% tokens have been generated and the remaining tokens are generated using different random seeds. Dark blue and light blue indicate the tokens that have been decoded and not decoded, respectively. The top row shows checkerboard pattern that distributes generated tokens (dark blue) uniformly throughout the image, yielding consistent generation results. The bottom row shows contiguous block pattern that concentrates generated tokens in the upper region, resulting in a diverse generation.
• Figure 4: Comparison of different token sampling strategies. The proposed GtR formulates the generation process as a two-stage checkerboard procedure: generation stage establishes global semantic structure through spatially non-adjacent tokens at conservative speed, followed by reconstruction stage completing remaining tokens within 1-2 steps through highly parallel generation.
• Figure 5: Qualitative comparison between Generation then Reconstruction (GtR) and the acceleration achieved through step reduction on MAR. GtR enables extreme acceleration while maintaining generation quality, whereas step reduction results in significant visual degradation.