HAS-VQ: Hessian-Adaptive Sparse Vector Quantization for High-Fidelity LLM Compression

TL;DR

HAS-VQ addresses the memory bandwidth bottleneck in LLM deployment by decoupling high-sensitivity outliers from weight distributions using Hessian-guided masking and then applying robust vector quantization to the dense body, complemented by a residual sparse feedback that exactly reconstructs outliers. The method minimizes Hessian-weighted loss, $\mathbb{E}[ (\mathbf{W}-\widehat{\mathbf{W}})^T \mathbf{H} (\mathbf{W}-\widehat{\mathbf{W}}) ]$, through a four-stage pipeline: Sensitivity Analysis, Sparse-Dense Decoupling, Robust VQ, and Residual Integration. On SmolLM2-1.7B-Instruct, HAS-VQ achieves a Pareto-dominant 4.23 BPP with PPL 14.23 (vs INT4’s 4.71 BPP and 20.03 PPL) and a High-Fidelity 7.03 BPP regime with PPL 10.12 nearly matching FP16 (10.04 PPL), corresponding to roughly 2.3× storage reduction. The approach is architecture-agnostic and well-suited for edge deployment due to substantial storage savings, offering a practical near-lossless compression pathway for bandwidth-constrained environments.

Abstract

Post-training quantization is essential for deploying Large Language Models (LLMs) on resource- constrained devices. However, standard integer quantization (e.g., INT4) fundamentally degrades per- formance by imposing a uniform grid on the heavy-tailed distribution of weight parameters, particularly in smaller-scale models (e.g., <2B parameters). We introduce HAS-VQ (Hessian-Adaptive Sparse Vec- tor Quantization), a compression framework that strictly decouples high-sensitivity outliers from the bulk weight distribution using second-order sensitivity analysis. HAS-VQ employs a Hessian-Masked Decoupling strategy to isolate sensitive parameters, followed by robust Vector Quantization (VQ) of the remaining dense body. Crucially, we introduce a residual sparse feedback mechanism that corrects quan- tization errors in the most sensitive dimensions, ensuring exact reconstruction of outliers. We evaluate HAS-VQ on SmolLM2-1.7B, demonstrating two distinct regimes of superiority: (1) Pareto Dominance over Integer Baselines: At 4.23 effective bits-per-parameter (BPP), we achieve a perplexity of 14.23, significantly outperforming the standard INT4 baseline (20.03 PPL at 4.71 BPP). (2) High-Fidelity Compression: Relative to the FP16 baseline, HAS-VQ achieves a 2.3x reduction in model size (7.03 BPP) while maintaining statistically indistinguishable perplexity (10.12 vs. 10.04), effectively offering a lossless compression alternative for bandwidth-constrained environments. The code is available at https://github.com/VladimerKhasia/HASVQ