Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CIMPool: Scalable Neural Network Acceleration for Compute-In-Memory using Weight Pools

Shurui Li, Puneet Gupta

TL;DR

This work tackles the SRAM CIM bottleneck by enabling much larger neural networks to fit on-chip without prohibitive accuracy loss. It introduces CIMPool, a hardware-aware framework that uses a fixed $128\times128$ weight pool with a 1-bit error term and structured pruning, coupled with dataflow optimizations and a specialized hardware scheduler to permute outputs efficiently. The key contributions include 1) a non-repeating weight-pool strategy ensuring full CIM utilization, 2) 1-bit error term with per-layer scaling, 3) binarized weight pool content with mean-value scaling, 4) structured error pruning and grouping to reduce overhead, and 5) a two-CIM-array architecture with optimized permutation and bit-serial computation. Empirically, CIMPool achieves up to $14.8\times$ compression with accuracy on par with 8-bit baselines on Food-101 and CIFAR datasets, reduces chip area by $62.3\%$ over iso-accuracy 4-bit CIM, and lowers total energy by up to $3.24\times$ when DRAM is used, enabling order-of-magnitude larger models within a fixed area budget. These results demonstrate a practical path to scalable, energy-efficient CIM acceleration for large neural networks across SRAM and DRAM storage options.

Abstract

Compute-in-memory (CIM) based neural network accelerators offer a promising solution to the Von Neumann bottleneck by computing directly within memory arrays. However, SRAM CIM faces limitations in executing larger models due to its cell size and on-chip memory constraints. This work proposes CIMPool, a CIM-aware compression and acceleration framework that counters this limitation through a weight sharing-based compression technique, aptly named `Weight Pool,' enabling significantly larger neural networks to be accommodated within on-chip memory constraints. This method minimizes the accuracy trade-off typically associated with parameter compression, allowing CIMPool to achieve a significantly larger compression ratio compared to the traditional quantization method with iso-accuracy. Furthermore, CIMPool co-optimizes the compression algorithm, hardware, and dataflow to efficiently implement the hardware permutation required by weight pool compression, with negligible area and throughput overhead. Empirical results demonstrate that CIMPool can achieve 8-bit level accuracy with an effective 0.5-bit precision, reduce chip area by 62.3% for ResNet-18, and enable the execution of an order of magnitude larger models for a given area budget in SRAM CIMs. When DRAM is used to store weights, CIMPool can reduce the total energy by 3.24x compared to iso-accuracy traditional CIMs.

CIMPool: Scalable Neural Network Acceleration for Compute-In-Memory using Weight Pools

TL;DR

This work tackles the SRAM CIM bottleneck by enabling much larger neural networks to fit on-chip without prohibitive accuracy loss. It introduces CIMPool, a hardware-aware framework that uses a fixed weight pool with a 1-bit error term and structured pruning, coupled with dataflow optimizations and a specialized hardware scheduler to permute outputs efficiently. The key contributions include 1) a non-repeating weight-pool strategy ensuring full CIM utilization, 2) 1-bit error term with per-layer scaling, 3) binarized weight pool content with mean-value scaling, 4) structured error pruning and grouping to reduce overhead, and 5) a two-CIM-array architecture with optimized permutation and bit-serial computation. Empirically, CIMPool achieves up to compression with accuracy on par with 8-bit baselines on Food-101 and CIFAR datasets, reduces chip area by over iso-accuracy 4-bit CIM, and lowers total energy by up to when DRAM is used, enabling order-of-magnitude larger models within a fixed area budget. These results demonstrate a practical path to scalable, energy-efficient CIM acceleration for large neural networks across SRAM and DRAM storage options.

Abstract

Compute-in-memory (CIM) based neural network accelerators offer a promising solution to the Von Neumann bottleneck by computing directly within memory arrays. However, SRAM CIM faces limitations in executing larger models due to its cell size and on-chip memory constraints. This work proposes CIMPool, a CIM-aware compression and acceleration framework that counters this limitation through a weight sharing-based compression technique, aptly named `Weight Pool,' enabling significantly larger neural networks to be accommodated within on-chip memory constraints. This method minimizes the accuracy trade-off typically associated with parameter compression, allowing CIMPool to achieve a significantly larger compression ratio compared to the traditional quantization method with iso-accuracy. Furthermore, CIMPool co-optimizes the compression algorithm, hardware, and dataflow to efficiently implement the hardware permutation required by weight pool compression, with negligible area and throughput overhead. Empirical results demonstrate that CIMPool can achieve 8-bit level accuracy with an effective 0.5-bit precision, reduce chip area by 62.3% for ResNet-18, and enable the execution of an order of magnitude larger models for a given area budget in SRAM CIMs. When DRAM is used to store weights, CIMPool can reduce the total energy by 3.24x compared to iso-accuracy traditional CIMs.

Paper Structure

This paper contains 32 sections, 4 equations, 11 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. SRAM CIM
  4. Weight Pool Compression
  5. Related Work
  6. CIMPool algorithm
  7. Non-repeating weight pool
  8. Error term
  9. Error term quantization
  10. Weight pool content and quantization
  11. Error pruning
  12. Error scaling factor
  13. Network mapping
  14. Overall framework flow
  15. CIMPool Architecture
  16. ...and 17 more sections

Figures (11)

  • Figure 1: Visualization of weight pool compression.
  • Figure 2: Visualization of z-dimension weight packing.
  • Figure 3: Accuracy vs. vector size of weight pool compression, for ResNet-18 on CIFAR-100.
  • Figure 4: (a) Zero masks of conventional semi-structured pruning, zero positions within the mask are not structured, but the mask is shared with a group of vectors to reduce storage overhead. (b), (c): Zero masks of our structured pruning approach with 50% and 75% sparsity.
  • Figure 5: (a): Training flow to generate the weight pool index and the error term. (b): Mapping generated weight pool content, index, and error term to CIMPool hardware.
  • ...and 6 more figures