QuantLRM: Quantization of Large Reasoning Models via Fine-Tuning Signals

TL;DR

QuantLRM proposes using fine-tuning traces, specifically weight-update signals from reasoning-focused training, to guide weight-only post-training quantization of large reasoning models. It defines channel importance via a novel two-ended, restricted-quadratic mapping that emphasizes both the smallest and largest weight updates, augmented by counting zero updates, and injects this signal into AWQ-style scaling with an exponent α optimized to minimize a quantization loss. Across SFT, DPO, and RL-finetuned LRMs on four reasoning benchmarks, QuantLRM delivers state-of-the-art 3-bit quantization performance, with notable gains on RL tasks (average ~6.55%) and meaningful improvements on other fine-tuning regimes, while maintaining a small calibration set and compatibility with existing inference engines. The method also demonstrates applicability when pre-finetuned checkpoints are unavailable via pseudo-fine-tuning and shows modest offline cost with comparable inference speed to baselines, highlighting the practical impact of leveraging fine-tuning dynamics for efficient model compression.

Abstract

Weight-only quantization is important for compressing Large Language Models (LLMs). Inspired by the spirit of classical magnitude pruning, we study whether the magnitude of weight updates during reasoning-incentivized fine-tuning can provide valuable signals for quantizing Large Reasoning Models (LRMs). We hypothesize that the smallest and largest weight updates during fine-tuning are more important than those of intermediate magnitude, a phenomenon we term "protecting both ends". Upon hypothesis validation, we introduce QuantLRM, which stands for weight quantization of LRMs via fine-tuning signals. We fit simple restricted quadratic functions on weight updates to protect both ends. By multiplying the average quadratic values with the count of zero weight updates of channels, we compute channel importance that is more effective than using activation or second-order information. We run QuantLRM to quantize various fine-tuned models (including supervised, direct preference optimization, and reinforcement learning fine-tuning) over four reasoning benchmarks (AIME-120, FOLIO, temporal sequences, and GPQA-Diamond) and empirically find that QuantLRM delivers a consistent improvement for LRMs quantization, with an average improvement of 6.55% on a reinforcement learning fine-tuned model. Also supporting non-fine-tuned LRMs, QuantLRM gathers effective signals via pseudo-fine-tuning, which greatly enhances its applicability.

Figures (3)

• Figure 1: Overview of channel importance computation. To protect weight updates $\Delta W$ on both ends of supervised (SFT), direct preference optimization (DPO), and reinforcement learning (RL) fine-tuning pipelines, we fit restricted quadratic functions on LRMs' weights and count the number of zero weight updates of each input channel. The input channel importance is the element-wise product of the average quadratic score and the count of zero updates. In practice, we increment the count of zero updates by $1$ for each channel, as discussed in \ref{['sec:our_method']}.
• Figure 2: Injecting channel importance $\mathbf{I}_{m\ell}$ and searching for optimal $\alpha$ to control the strength of scaling. QuantLRM finds the best scaling factors that incorporate fine-tuning signals.
• Figure 3: Performance of 3-bit QuantLRM at different training steps. As training progresses, QuantLRM’s performance improves.