Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LegoNet: Memory Footprint Reduction Through Block Weight Clustering

Joseph Bingham, Noah Green, Saman Zonouz

TL;DR

This work proposes LegoNet, a compression technique that constructs blocks of weights of the entire model regardless of layer type and clusters these induced blocks, and was able to compress ResNet-50 trained for Cifar-10 and ImageNet with only 32 4x4 blocks with less than 3% accuracy loss.

Abstract

As the need for neural network-based applications to become more accurate and powerful grows, so too does their size and memory footprint. With embedded devices, whose cache and RAM are limited, this growth hinders their ability to leverage state-of-the-art neural network architectures. In this work, we propose \textbf{LegoNet}, a compression technique that \textbf{constructs blocks of weights of the entire model regardless of layer type} and clusters these induced blocks. Using blocks instead of individual values to cluster the weights, we were able to compress ResNet-50 trained for Cifar-10 and ImageNet with only 32 4x4 blocks, compressing the memory footprint by over a factor of \textbf{64x without having to remove any weights} or changing the architecture and \textbf{no loss to accuracy}, nor retraining or any data, and show how to find an arrangement of 16 4x4 blocks that gives a compression ratio of \textbf{128x with less than 3\% accuracy loss}. This was all achieved with \textbf{no need for (re)training or fine-tuning}.

LegoNet: Memory Footprint Reduction Through Block Weight Clustering

TL;DR

This work proposes LegoNet, a compression technique that constructs blocks of weights of the entire model regardless of layer type and clusters these induced blocks, and was able to compress ResNet-50 trained for Cifar-10 and ImageNet with only 32 4x4 blocks with less than 3% accuracy loss.

Abstract

As the need for neural network-based applications to become more accurate and powerful grows, so too does their size and memory footprint. With embedded devices, whose cache and RAM are limited, this growth hinders their ability to leverage state-of-the-art neural network architectures. In this work, we propose \textbf{LegoNet}, a compression technique that \textbf{constructs blocks of weights of the entire model regardless of layer type} and clusters these induced blocks. Using blocks instead of individual values to cluster the weights, we were able to compress ResNet-50 trained for Cifar-10 and ImageNet with only 32 4x4 blocks, compressing the memory footprint by over a factor of \textbf{64x without having to remove any weights} or changing the architecture and \textbf{no loss to accuracy}, nor retraining or any data, and show how to find an arrangement of 16 4x4 blocks that gives a compression ratio of \textbf{128x with less than 3\% accuracy loss}. This was all achieved with \textbf{no need for (re)training or fine-tuning}.
Paper Structure (11 sections, 5 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 11 sections, 5 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Motivation
  3. Background and Related Work
  4. Methods
  5. Design
  6. Theoretical Analysis
  7. Results
  8. Setup
  9. Experimental Results
  10. Discussion
  11. Conclusion

Figures (4)

  • Figure 1: An overview of LegoNet algorithm. First a trained model's weight matrices are chunked into blocks, or Legos. Then the Legos are clustered into groups. Then the blocks are replaced by the index of the cluster they belong to. At inference time, the centroid representative of this cluster is used as the weight values.
  • Figure 2: A comparison of memory consumption (blue bars) and the size of memory available in the STM32F7 micro-controller (red dotted line). VGG16 is compared to compressed versions, such as (1) Basis Li_2019_ICCV, (2) MBA DBLP:journals/corr/abs-2006-03669, (3) CRELU DBLP:journals/corr/abs-2006-03669, (4) CIRC DBLP:journals/corr/abs-2006-03669 and our method, LegoNet (5). Even though others may reduce other attributes (runtime, FLOPs, etc.) only LegoNet gets the model small enough to fit into the main memory.
  • Figure 3: Analysis of $K$ values for a given $b$ value, $M$ and dataset. Note, $b = 4$ is the best for almost all models used in terms of compression ratio for a given accuracy tolerance.
  • Figure 4: Accuracy for a variety of $K$ values (orange line) compared to the baseline (blue line).