Column-wise Quantization of Weights and Partial Sums for Accurate and Efficient Compute-In-Memory Accelerators

TL;DR

This paper tackles the ADC-related bottlenecks in compute-in-memory DNN accelerators by aligning weight and partial-sum quantization at the column level, enabling finer control without increasing dequantization overhead. It introduces a CIM-oriented convolution framework with novel tiling and group convolution to support fine-grained, column-wise quantization, and enables one-stage quantization-aware training using LSQ-scale factors for both weights and partial-sums. Empirical results on ResNet-20 (CIFAR-10/100) and ResNet-18 (ImageNet) show consistent accuracy gains over prior methods and robustness to memory-cell variations. The approach offers practical improvements for hardware-efficient CIM DNNs and is accompanied by an open-source framework for broader adoption.

Abstract

Compute-in-memory (CIM) is an efficient method for implementing deep neural networks (DNNs) but suffers from substantial overhead from analog-to-digital converters (ADCs), especially as ADC precision increases. Low-precision ADCs can reduce this overhead but introduce partial-sum quantization errors degrading accuracy. Additionally, low-bit weight constraints, imposed by cell limitations and the need for multiple cells for higher-bit weights, present further challenges. While fine-grained partial-sum quantization has been studied to lower ADC resolution effectively, weight granularity, which limits overall partial-sum quantized accuracy, remains underexplored. This work addresses these challenges by aligning weight and partial-sum quantization granularities at the column-wise level. Our method improves accuracy while maintaining dequantization overhead, simplifies training by removing two-stage processes, and ensures robustness to memory cell variations via independent column-wise scale factors. We also propose an open-source CIM-oriented convolution framework to handle fine-grained weights and partial-sums efficiently, incorporating a novel tiling method and group convolution. Experimental results on ResNet-20 (CIFAR-10, CIFAR-100) and ResNet-18 (ImageNet) show accuracy improvements of 0.99%, 2.69%, and 1.01%, respectively, compared to the best-performing related works. Additionally, variation analysis reveals the robustness of our method against memory cell variations. These findings highlight the effectiveness of our quantization scheme in enhancing accuracy and robustness while maintaining hardware efficiency in CIM-based DNN implementations. Our code is available at https://github.com/jiyoonkm/ColumnQuant.

Figures (10)

• Figure 1: Overview of the proposed quantization method and previous works.
• Figure 2: Implementation of the convolution layer on bit-scalable CIM architecture. (a) Im2col mapping and tiling process. $C_{in}$ represents the number of input channels, and $K$ denotes the kernel size. (b) Bit-scalable CIM architecture.
• Figure 3: Matrix multiplication of a DNN layer with the proposed column-wise weight and partial-sum quantization. ($A$: activations, $W$: weights, $P_i$: partial-sums, $s_{w_i}$: scaling factor for weights, $s_{p_i}$: scaling factors for partial-sums)
• Figure 4: Dequantization process after matrix multiplication in CIM. From (a) to (c), weights are quantized layer-wise, with partial-sums quantized (a) layer-wise, (b) array-wise, or (c) column-wise. (d) shows our proposed column-wise weight and partial-sum quantization. This figure assumes that the 4-bit weights are implemented with two 2-bit cells. Memory cells with the same background color belong to the same output channel.
• Figure 5: Convolution framework overview for bit-scalable CIM. The figure outlines the process, starting with weight duplication and quantization into bit-splits. Each bit-split undergoes array-wise convolution with weight mapping, tiling, and partial-sum quantization. Finally, outputs from each split are bit-shifted and accumulated to form the final convolution output.