Binary Quadratic Quantization: Beyond First-Order Quantization for Real-Valued Matrix Compression

TL;DR

Binary Quadratic Quantization (BQQ) introduces a second-order, binary-basis approach to real-valued matrix compression by representing a matrix as a sum of binary matrix products with independent scalings: $\mathbf{W} \approx \sum_{i=0}^{p-1} (r_i \mathbf{Y}_i \mathbf{Z}_i + s_i \mathbf{Y}_i \mathbf{1}_Z + t_i \mathbf{1}_Y \mathbf{Z}_i) + u \mathbf{1}$. It builds on Polynomial Unconstrained Binary Optimization (PUBO) and Annealed Mean Field Descent (AMFD) to solve the resulting NP-hard subproblems via a greedy, alternating optimization strategy, enabling efficient weight quantization and matrix compression. The authors demonstrate state-of-the-art results in matrix data compression across diverse datasets and achieve top-tier post-training quantization (PTQ) performance for Vision Transformers, including data-free PTQ with 2-bit-equivalent models and substantial memory savings. The work highlights practical hardware implications, such as addition-only inference and reduced parameter overhead through group-wise scaling, suggesting broad applicability to retrieval systems and large-scale learning on massive data. Overall, BQQ shows that second-order binary representations can capture complex matrix structure more effectively than first-order quantization while remaining highly storage-efficient.

Abstract

This paper proposes a novel matrix quantization method, Binary Quadratic Quantization (BQQ). In contrast to conventional first-order quantization approaches, such as uniform quantization and binary coding quantization, that approximate real-valued matrices via linear combinations of binary bases, BQQ leverages the expressive power of binary quadratic expressions while maintaining an extremely compact data format. We validate our approach with two experiments: a matrix compression benchmark and post-training quantization (PTQ) on pretrained Vision Transformer-based models. Experimental results demonstrate that BQQ consistently achieves a superior trade-off between memory efficiency and reconstruction error than conventional methods for compressing diverse matrix data. It also delivers strong PTQ performance, even though we neither target state-of-the-art PTQ accuracy under tight memory constraints nor rely on PTQ-specific binary matrix optimization. For example, our proposed method outperforms the state-of-the-art PTQ method by up to 2.2\% and 59.1% on the ImageNet dataset under the calibration-based and data-free scenarios, respectively, with quantization equivalent to 2 bits. These findings highlight the surprising effectiveness of binary quadratic expressions for efficient matrix approximation and neural network compression.

Figures (4)

• Figure 1: Comparison between BQQ and first-order quantization in a DNN layer.
• Figure 2: Weight scaling methods.
• Figure 3: Comparison of the trade-off between reconstruction error (MSE) and memory size for five matrix datasets. Rows 1–4 show the trade-off curves, value distributions, singular value distributions, and 3D plots of the matrices, respectively.
• Figure S.1: Effect of the number of optimization steps $N_\text{step}$ (top) and total quantization time (bottom) on final ImageNet top-1 accuracy in the data-free quantization setting.

Theorems & Definitions (1)

• proof