TSB: Tiny Shared Block for Efficient DNN Deployment on NVCIM Accelerators

TL;DR

The paper tackles accuracy and deployment challenges caused by device variation in non-volatile CIM (NVCIM) accelerators by introducing Tiny Shared Block (TSB), a small shared $1\times1$ convolution that is inserted into DNN backbones. After pretraining to an early convergence, TSB learns to suppress variation-induced feature noise and is deployed with write-verify on a tiny subset of weights, leveraging shared weights across layers to minimize training and hardware overhead. Empirical results across LeNet-3, VGG-8, and ResNet-18 show substantial gains: inference-accuracy gaps shrink by over $20\times$, training time reduces by over $5\times$, and write-verify overhead falls below $0.4\%$ of original weights, with NeuroSim-based hardware overheads remaining modest. The approach is compatible with existing CIM architectures and demonstrates a practical path to robust, energy-efficient DNN deployment on NVCIM accelerators.

Abstract

Compute-in-memory (CIM) accelerators using non-volatile memory (NVM) devices offer promising solutions for energy-efficient and low-latency Deep Neural Network (DNN) inference execution. However, practical deployment is often hindered by the challenge of dealing with the massive amount of model weight parameters impacted by the inherent device variations within non-volatile computing-in-memory (NVCIM) accelerators. This issue significantly offsets their advantages by increasing training overhead, the time and energy needed for mapping weights to device states, and diminishing inference accuracy. To mitigate these challenges, we propose the "Tiny Shared Block (TSB)" method, which integrates a small shared 1x1 convolution block into the DNN architecture. This block is designed to stabilize feature processing across the network, effectively reducing the impact of device variation. Extensive experimental results show that TSB achieves over 20x inference accuracy gap improvement, over 5x training speedup, and weights-to-device mapping cost reduction while requiring less than 0.4% of the original weights to be write-verified during programming, when compared with state-of-the-art baseline solutions. Our approach provides a practical and efficient solution for deploying robust DNN models on NVCIM accelerators, making it a valuable contribution to the field of energy-efficient AI hardware.

Figures (4)

• Figure 1: The tiny shared block deployment design.
• Figure 2: Workflow of TSB deployment method.
• Figure 3: Normalized accuracy distances (the lower the batter) between inference accuracy and the ideal accuracy across different backbone device variation $\sigma_{d}$: (a) LeNet-3 on MNIST, (b) VGG-8 on CIFAR-10, and (c) ResNet-18 on Tiny ImageNet.
• Figure 4: Convergence process of the VGG-8 model with and without the TSB method applied.