Activation Sensitivity as a Unifying Principle for Post-Training Quantization

TL;DR

This work proposes activation sensitivity as a unifying principle for post-training quantization (PTQ) of large language models. By deriving sensitivity from a first-order Taylor expansion, it defines a loss-aware saliency measure based on the gradient-weighted activations, linking activation-magnitude and covariance-based perspectives and clarifying when AWQ and GPTQ approximate the same underlying quantity. It analyzes the design space of sensitivity metrics (first-order, second-order, Fisher-based) and their granularity, highlighting the limitations of static, layer-local proxy objectives due to cross-layer error accumulation and calibration distribution shifts. The paper also situates PTQ within classical second-order compression theory, explaining why direct transfers from pruning theory are imperfect for dense, non-retraining quantization, and outlines open challenges and directions for future research, including cross-layer sensitivity, adaptive estimation, and beyond-reconstruction objectives. Overall, it provides a principled conceptual framework for understanding and comparing PTQ methods through the lens of activation sensitivity, without proposing a new algorithm.

Abstract

Post-training quantization (PTQ) methods for large language models rely on heuristics that implicitly estimate which weight channels most strongly influence model behavior. Two dominant paradigms have emerged: activation-aware methods such as AWQ prioritize channels with large activation magnitudes, while second-order methods such as GPTQ allocate quantization error according to input covariance structure. Despite strong empirical performance, these approaches remain conceptually fragmented, and it is unclear what underlying quantity they are approximating. In this work, we present a unified theoretical framework for PTQ by formalizing activation sensitivity, defined as the expected impact of channel-wise perturbations on the loss. Using a first-order Taylor expansion, we show that sensitivity naturally arises as the squared norm of gradient-weighted activations, yielding a principled measure of channel importance that captures both activation magnitude and downstream error propagation. Within this framework, AWQ and GPTQ can be interpreted as complementary approximations that recover sensitivity under distinct simplifying assumptions. We analyze the design space of sensitivity metrics, connect gradient-based saliency, Fisher information, and Hessian-based criteria, and clarify their relationships to classical pruning methods such as Optimal Brain Damage and Optimal Brain Surgeon. Rather than proposing a new quantization algorithm, this work provides a conceptual foundation for understanding and comparing post-training quantization methods through the lens of sensitivity.