Low-bit Model Quantization for Deep Neural Networks: A Survey

TL;DR

This survey analyzes the rapid evolution of low-bit quantization in deep neural networks, framing the problem as mapping floating-point weights/activations to low-precision representations while balancing accuracy, latency, and storage. It classifies state-of-the-art methods into eight major categories and twenty-four subcategories, detailing fundamental concepts, practical techniques, and advances across topics such as scale/zero-point optimization, mixed precision, data-free strategies, and hardware-aware formats. The authors summarize about 179 papers, discuss key methodological trends (PTQ vs QAT, reconstruction losses, distribution reshaping, and diffusion-model considerations), and highlight open research opportunities, including multimodal deployment, hybrid compression, and co-optimization with hardware. A GitHub resource with curated model quantization methods is provided to assist practitioners and researchers in selecting and implementing suitable techniques. The work thus offers a comprehensive, technology-agnostic roadmap for improving the efficiency and practicality of quantized DNNs in real-world settings.

Abstract

With unprecedented rapid development, deep neural networks (DNNs) have deeply influenced almost all fields. However, their heavy computation costs and model sizes are usually unacceptable in real-world deployment. Model quantization, an effective weight-lighting technique, has become an indispensable procedure in the whole deployment pipeline. The essence of quantization acceleration is the conversion from continuous floating-point numbers to discrete integer ones, which significantly speeds up the memory I/O and calculation, i.e., addition and multiplication. However, performance degradation also comes with the conversion because of the loss of precision. Therefore, it has become increasingly popular and critical to investigate how to perform the conversion and how to compensate for the information loss. This article surveys the recent five-year progress towards low-bit quantization on DNNs. We discuss and compare the state-of-the-art quantization methods and classify them into 8 main categories and 24 sub-categories according to their core techniques. Furthermore, we shed light on the potential research opportunities in the field of model quantization. A curated list of model quantization is provided at https://github.com/Kai-Liu001/Awesome-Model-Quantization.

Figures (8)

• Figure 1: Illustration of different quantization schemes, including symmetric and asymmetric quantization, uniform and non-uniform quantization.
• Figure 2: QAT usually takes large datasets to retrain the weights and quantizers' parameters, while PTQ mostly only leverages the calibration data to obtain the quantizer parameters.
• Figure 3: Distribution of activation and weight in common models. For CNN, the distributions of activations and weights are mostly symmetrical and similar to normal distribution. For ViT, the situation is different. Both activation and weights show obvious correlations along the channel.
• Figure 4: Taxonomy for recent quantization methods. We classify 179 papers into 8 categories and further into 24 sub-categories.
• Figure 5: Two paradigms of mixed precision. The left one is input irrelevant, and the bit-width is allocated during the calibration process. The right one is input relevant, where a bit controller processes the image first and then dynamically decides the bit-width allocation.