MixAR: Mixture Autoregressive Image Generation

TL;DR

MixAR tackles the fidelity bottleneck of discrete autoregressive image generation by introducing a factorized framework that leverages discrete tokens as priors for continuous autoregression. It proposes three discrete–continuous guidance strategies (DC-Mix, DC-SA, DC-CA) and a Training-Inference Mixture (TI-Mix) to align training and inference distributions, supported by a MAR-based backbone with discrete tokenization and a diffusion refinement head. Empirically, MixAR delivers stronger fidelity than traditional MAR and discrete baselines on ImageNet-256, with DC-Mix offering the best efficiency–fidelity balance and TI-Mix further boosting generation quality. The work demonstrates the practical impact of discrete priors in guiding high-fidelity continuous generation, enabling parameter-efficient models that scale in real-world image synthesis tasks, while highlighting limitations around pretrained tokenizers and opportunities for joint tokenizer optimization.

Abstract

Autoregressive (AR) approaches, which represent images as sequences of discrete tokens from a finite codebook, have achieved remarkable success in image generation. However, the quantization process and the limited codebook size inevitably discard fine-grained information, placing bottlenecks on fidelity. Motivated by this limitation, recent studies have explored autoregressive modeling in continuous latent spaces, which offers higher generation quality. Yet, unlike discrete tokens constrained by a fixed codebook, continuous representations lie in a vast and unstructured space, posing significant challenges for efficient autoregressive modeling. To address these challenges, we introduce MixAR, a novel framework that leverages mixture training paradigms to inject discrete tokens as prior guidance for continuous AR modeling. MixAR is a factorized formulation that leverages discrete tokens as prior guidance for continuous autoregressive prediction. We investigate several discrete-continuous mixture strategies, including self-attention (DC-SA), cross-attention (DC-CA), and a simple approach (DC-Mix) that replaces homogeneous mask tokens with informative discrete counterparts. Moreover, to bridge the gap between ground-truth training tokens and inference tokens produced by the pre-trained AR model, we propose Training-Inference Mixture (TI-Mix) to achieve consistent training and generation distributions. In our experiments, we demonstrate a favorable balance of the DC-Mix strategy between computational efficiency and generation fidelity, and consistent improvement of TI-Mix.

Figures (8)

• Figure 1: Intuition behind MixAR. Compared with the traditional masked autoregressive (MAR) strategy that predicts the full image from only a subset of visible patches (a), MixAR reconstructs the image from a partially degraded yet still informative version (b). This design provides richer contextual cues and captures fine-grained details more effectively.
• Figure 2: MixAR vs Maskbit Top: Images generated by Maskbit maskbit. Bottom: Images generated by our MixAR-L model conditioned on the discrete tokens produced by Maskbit, demonstrating significant improvements in generation quality.
• Figure 3: Overview of the training and inference pipeline of MixAR. During training, we first mix ground truth continuous and discrete tokens with DC-Mix, followed by TI-Mix that replaces a certain portion of ground truth discrete tokens with generated ones. The continuous AR model then predicts the ground-truth continuous tokens given the mixed tokens, optimized by diffusion loss. During inference, we first generate discrete tokens with a pretrained autoregressive model and use them to guide the continuous autoregressive model to generate more information-rich continuous tokens.
• Figure 4: Comparison between MAR and DC-Mix. (a) In MAR, masked positions are replaced by a single meaningless mask token before prediction. (b) In contrast, DC-Mix pads masked regions with informative discrete tokens that carry semantic and structural cues, making the prediction easier with no additional computation cost. The legend follows the same convention as Figure \ref{['fig:pipeline']}.
• Figure 5: Illustration of TI-Mix. As shown in Figure \ref{['fig:dcmix_vs_mar']}, without TI-Mix, only ground truth discrete tokens are used as guidance during training, which is different from the generated tokens used in inference. In contrast, TI-Mix uses a mixture of ground truth and generated discrete tokens as guidance during training, effectively mitigating the distribution gap. Here the positions 3,4,5 form the masked region shared by ground truth discrete and continuous tokens. The legend follows the same convention as Figure \ref{['fig:pipeline']}.