What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study

TL;DR

Is a systematic empirical study of quantization-aware training (QAT) for reasoning-focused LLMs to address large accuracy losses from post-training quantization at extreme low-bit settings. The authors compare training objectives (KD vs SFT), PTQ initialization, RL integration, and data strategies, culminating in a three-stage Reasoning-QAT workflow. KD consistently outperforms SFT as a recovery objective, PTQ initialization accelerates convergence, and KD-cold-started RL via GRPO yields further gains when training data align with calibration domains. Across multiple backbones and reasoning benchmarks, Reasoning-QAT outperforms state-of-the-art PTQ baselines and demonstrates substantial gains even in the 2-bit regime, enabling practical low-bit reasoning deployments.

Abstract

Reasoning models excel at complex tasks such as coding and mathematics, yet their inference is often slow and token-inefficient. To improve the inference efficiency, post-training quantization (PTQ) usually comes with the cost of large accuracy drops, especially for reasoning tasks under low-bit settings. In this study, we present a systematic empirical study of quantization-aware training (QAT) for reasoning models. Our key findings include: (1) Knowledge distillation is a robust objective for reasoning models trained via either supervised fine-tuning or reinforcement learning; (2) PTQ provides a strong initialization for QAT, improving accuracy while reducing training cost; (3) Reinforcement learning remains feasible for quantized models given a viable cold start and yields additional gains; and (4) Aligning the PTQ calibration domain with the QAT training domain accelerates convergence and often improves the final accuracy. Finally, we consolidate these findings into an optimized workflow (Reasoning-QAT), and show that it consistently outperforms state-of-the-art PTQ methods across multiple LLM backbones and reasoning datasets. For instance, on Qwen3-0.6B, it surpasses GPTQ by 44.53% on MATH-500 and consistently recovers performance in the 2-bit regime.

Figures (6)

• Figure 1: The performance degradation by post-training quantization on reasoning and non-reasoning tasks. We adopt GPTQ with 3-bit weight only quantization with group size 128, and the results are based on DeepSeek-R1-Distill-Qwen-1.5B and Qwen-4B.
• Figure 2: PTQ initialization improves QAT efficiency (RTN+KD vs. GPTQ+KD on MATH-500): (a) accuracy, (b) loss.
• Figure 3: KD vs. SFT under GPTQ initialization on MATH-500.
• Figure 4: RL effects after KD cold-start (W3G128). (a) reward$\uparrow$ with length$\downarrow$, (b) entropy$\downarrow$, (c) test accuracy$\uparrow$ with length$\downarrow$.
• Figure 5: Comparison with different combinations of PTQ and QAT data domain. We present test accuracy curves on MATH-500 for the R1-Qwen-1.5B model under W3G128 setting.