OMPQ: Orthogonal Mixed Precision Quantization

TL;DR

OMPQ introduces an Orthogonality Metric (ORM) as a fast proxy for the mixed-precision quantization objective, enabling efficient bit-width allocation without extensive searching. By deconstructing networks into functions and estimating pairwise layer orthogonality via Monte Carlo sampling, ORM feeds a simple linear program that assigns bits under a model-size constraint, compatible with both QAT and PTQ. Empirical results on ImageNet demonstrate state-of-the-art or competitive accuracy with orders-of-magnitude reductions in data and search requirements, confirmed on ResNet-18 and MobileNetV2. The method provides actionable insights into layer-wise bit allocation driven by orthogonality, and offers a plug-and-play quantization component with practical speedups.

Abstract

To bridge the ever increasing gap between deep neural networks' complexity and hardware capability, network quantization has attracted more and more research attention. The latest trend of mixed precision quantization takes advantage of hardware's multiple bit-width arithmetic operations to unleash the full potential of network quantization. However, this also results in a difficult integer programming formulation, and forces most existing approaches to use an extremely time-consuming search process even with various relaxations. Instead of solving a problem of the original integer programming, we propose to optimize a proxy metric, the concept of network orthogonality, which is highly correlated with the loss of the integer programming but also easy to optimize with linear programming. This approach reduces the search time and required data amount by orders of magnitude, with little compromise on quantization accuracy. Specifically, we achieve 72.08% Top-1 accuracy on ResNet-18 with 6.7Mb, which does not require any searching iterations. Given the high efficiency and low data dependency of our algorithm, we used it for the post-training quantization, which achieve 71.27% Top-1 accuracy on MobileNetV2 with only 1.5Mb. Our code is available at https://github.com/MAC-AutoML/OMPQ.

Figures (11)

• Figure 1: Comparison of the resources used to obtain the optimal bit configuration between our algorithm and other mixed precision algorithms (FracBits yang2020fracbits, HAWQ dong2019hawq, BRECQ li2021brecq) on ResNet-$18$. "Searching Data" is the number of input images.
• Figure 2: Overview. Left: Deconstruct the model into a set of functions $\mathcal{F}$. Middle: ORM symmetric matrix calculated from $\mathcal{F}$. Right: Linear programming problem constructed by the importance factor $\theta$ to derive optimal bit configuration.
• Figure 3: Relationship between orthogonality and accuracy for different quantization configurations on ResNet-$18$ and MobileNetV$2$.
• Figure 4: Mixed precision quantization comparison of OMPQ and BRECQ on ResNet-$18$ and MobileNetV$2$.
• Figure 5: The former $i$ layers deconstruction(left) and the $i$-th layer deconstruction(right). We omit the BatchNorm layer ioffe2015batch and the activation layer for simplicity while $X_1=X, X_2=g_1(X_1), \dots, X_L=g_{L-1}(X_{L-1})$.

Theorems & Definitions (1)

• proof