Computing-In-Memory Aware Model Adaption For Edge Devices
Ming-Han Lin, Tian-Sheuan Chang
TL;DR
The paper tackles throughput and accuracy bottlenecks in computing-in-memory (CIM) edge accelerators caused by limited macro size and ADC precision. It introduces a two-stage CIM-aware model adaptation: first, CIM Aware Morphing (inspired by MorphNet) prunes and expands channel counts to fit CIM macro constraints; second, ADC Aware Learned Scaling performs two-phase quantization-aware training to align weights and partial sums with 4-bit weights and 5-bit partial sums, simulating CIM behavior. The approach yields high CIM macro utilization (≈90%), concurrent activation of up to 256 wordlines, and compression up to ≈93% while preserving accuracy close to baseline, with substantial reductions in weight-loading and computation latency. This framework demonstrates practical gains for edge DL by tightly coupling neural network adaptation with CIM hardware constraints, outperforming prior methods in macro utilization, compression, and speedups.
Abstract
Computing-in-Memory (CIM) macros have gained popularity for deep learning acceleration due to their highly parallel computation and low power consumption. However, limited macro size and ADC precision introduce throughput and accuracy bottlenecks. This paper proposes a two-stage CIM-aware model adaptation process. The first stage compresses the model and reallocates resources based on layer importance and macro size constraints, reducing model weight loading latency while improving resource utilization and maintaining accuracy. The second stage performs quantization-aware training, incorporating partial sum quantization and ADC precision to mitigate quantization errors in inference. The proposed approach enhances CIM array utilization to 90\%, enables concurrent activation of up to 256 word lines, and achieves up to 93\% compression, all while preserving accuracy comparable to previous methods.