Mixed-Precision Training and Compilation for RRAM-based Computing-in-Memory Accelerators
Rebecca Pelke, Joel Klein, Jose Cubero-Cascante, Nils Bosbach, Jan Moritz Joseph, Rainer Leupers
TL;DR
This work targets the latency-accuracy gap in CIM accelerators caused by limited bit precision. It introduces a mixed-precision quantization framework (MPQ) and a CIM-aware compiler, powered by reinforcement learning (CIM-AQ), to automatically select quantization per layer; it is complemented by a TVM-based CIM compiler that maps MPQ models to crossbar-based hardware with CIM-specific optimizations. The approach achieves up to 2.48× speedup on ImageNet models (best case for VGG-16) with only a 0.086% accuracy loss, supported by a robust training-and-compile pipeline and constraint strategies that prune the search space. This combination significantly enhances CIM efficiency for DNN workloads and provides a scalable path toward practical, low-power, high-throughput ML in-memory computing.
Abstract
Computing-in-Memory (CIM) accelerators are a promising solution for accelerating Machine Learning (ML) workloads, as they perform Matrix-Vector Multiplications (MVMs) on crossbar arrays directly in memory. Although the bit widths of the crossbar inputs and cells are very limited, most CIM compilers do not support quantization below 8 bit. As a result, a single MVM requires many compute cycles, and weights cannot be efficiently stored in a single crossbar cell. To address this problem, we propose a mixed-precision training and compilation framework for CIM architectures. The biggest challenge is the massive search space, that makes it difficult to find good quantization parameters. This is why we introduce a reinforcement learning-based strategy to find suitable quantization configurations that balance latency and accuracy. In the best case, our approach achieves up to a 2.48x speedup over existing state-of-the-art solutions, with an accuracy loss of only 0.086 %.