Optimizing Deep Neural Networks using Safety-Guided Self Compression

TL;DR

The paper tackles deploying deep networks on resource-limited devices by introducing safety-driven self-compression that prunes and quantizes weights under the guidance of a preservation set. It develops a differentiable quantization mechanism with a learnable bit-depth, guided by Grad-CAM, uncertainty sampling, and clustering, integrated into a joint loss that balances accuracy, sparsity, and preservation. Experiments on a CNN for MNIST and a Transformer decoder show up to a 2.5% test-accuracy gain with model size reduced to about 60% of the original, accompanied by improved generalization and reduced variance. The framework offers a practical, robust pathway for reliable, hardware-aware model compression, with code available on GitHub.

Abstract

The deployment of deep neural networks on resource-constrained devices necessitates effective model com- pression strategies that judiciously balance the reduction of model size with the preservation of performance. This study introduces a novel safety-driven quantization framework that leverages preservation sets to systematically prune and quantize neural network weights, thereby optimizing model complexity without compromising accuracy. The proposed methodology is rigorously evaluated on both a convolutional neural network (CNN) and an attention-based language model, demonstrating its applicability across diverse architectural paradigms. Experimental results reveal that our framework achieves up to a 2.5% enhancement in test accuracy relative to the original unquantized models while maintaining 60% of the initial model size. In comparison to conventional quantization techniques, our approach not only augments generalization by eliminating parameter noise and retaining essential weights but also reduces variance, thereby ensuring the retention of critical model features. These findings underscore the efficacy of safety-driven quantization as a robust and reliable strategy for the efficient optimization of deep learn- ing models. The implementation and comprehensive experimental evaluations of our framework are publicly accessible at GitHub.

Figures (3)

• Figure 1: (a) The solid black line represents the best fit to the data, showcasing how the model retains its predictive capacity, while the gray dots represent results obtained using our proposed method. As the parameters are pruned, the network's size is reduced rapidly during the early stages of training, followed by further compression. Importantly, this approach preserves test performance, maintaining accuracy while preventing any significant increase in variance. (b) The plot illustrates how training time accelerates as parameters are pruned from the network. As the number of iterations increases, the network becomes more efficient, leading to a significant reduction in training time (sampled 3000 data points).
• Figure 2: Flowchart depicting the creation of the preservation set for CNN and Transformer models: Grad-CAM, uncertainty sampling, and projection at the last layer ensure diversity, with the preservation set guiding quantization.
• Figure 3: Histogram with density curve of neural network weights post-quantization. The distribution shows a concentration of weights near central values, indicating effective pruning and quantization of less impactful weights. The tails represent significant weights preserved to maintain essential model features, supporting generalization and reducing variance. This distribution underscores the effectiveness of safety-driven quantization in achieving efficient model compression without sacrificing accuracy.