PTQ4ARVG: Post-Training Quantization for AutoRegressive Visual Generation Models

TL;DR

This work addresses the challenging problem of post-training quantization for AutoRegressive Visual Generation (ARVG) models, which combine fixed-length token sequences with AdaLN conditioning. It introduces a training-free framework, PTQ4ARVG, built from three components: Gain-Projected Scaling (GPS) to optimally scale activations and weights via a Taylor-series-based loss analysis; Static Token-Wise Quantization (STWQ) to offline-parameterize token-wise quantization leveraging position-invariant distributions; and Distribution-Guided Calibration (DGC) to select calibration samples by distribution entropy. Together, these components enable reliable 6- or 8-bit quantization across VAR, RAR, PAR, and MAR models with competitive image-quality metrics and practical speedups, while avoiding online calibration or customized kernels. The results demonstrate substantial improvements over existing training-free and training-based baselines, and the approach provides a concrete pathway for deploying ARVG models in resource-constrained environments. The work also offers a theoretically grounded framework that could generalize to other temporal and autoregressive vision models.

Abstract

AutoRegressive Visual Generation (ARVG) models retain an architecture compatible with language models, while achieving performance comparable to diffusion-based models. Quantization is commonly employed in neural networks to reduce model size and computational latency. However, applying quantization to ARVG remains largely underexplored, and existing quantization methods fail to generalize effectively to ARVG models. In this paper, we explore this issue and identify three key challenges: (1) severe outliers at channel-wise level, (2) highly dynamic activations at token-wise level, and (3) mismatched distribution information at sample-wise level. To these ends, we propose PTQ4ARVG, a training-free post-training quantization (PTQ) framework consisting of: (1) Gain-Projected Scaling (GPS) mitigates the channel-wise outliers, which expands the quantization loss via a Taylor series to quantify the gain of scaling for activation-weight quantization, and derives the optimal scaling factor through differentiation.(2) Static Token-Wise Quantization (STWQ) leverages the inherent properties of ARVG, fixed token length and position-invariant distribution across samples, to address token-wise variance without incurring dynamic calibration overhead.(3) Distribution-Guided Calibration (DGC) selects samples that contribute most to distributional entropy, eliminating the sample-wise distribution mismatch. Extensive experiments show that PTQ4ARVG can effectively quantize the ARVG family models to 8-bit and 6-bit while maintaining competitive performance. Code is available at http://github.com/BienLuky/PTQ4ARVG .

Figures (17)

• Figure 1: Challenges of ARVG quantization. (a) Severe outliers at channel-wise level. (b)(c) Highly dynamic activations at token-wise level. (d) Mismatched distribution information at sample-wise level. Data from the RAR-B block "blocks.23".
• Figure 2: Overview of ARVG models.(Left) The autoregressive architecture, mechanism, and challenges of ARVG. (Right) Existing ARVG models with different token prediction granularity.
• Figure 3: When $R_x^i > R_x^j$, the statistical results of Remark 1.
• Figure 4: Inputs of AdaLN in RAR-B from different sample class. The distribution remains invariant across samples.
• Figure 5: Visualizing the Advantages of GPS on VAR-d16 at 6-bit precision.