In-Pipeline Integration of Digital In-Memory-Computing into RISC-V Vector Architecture to Accelerate Deep Learning

TL;DR

The paper tackles the data-movement bottleneck in edge DL by tightly integrating a Digital In-Memory Computing (DIMC) tile into a RISC-V vector core. It introduces a four-instruction custom ISA to control data loading, in-memory MAC operations, and write-back, enabling efficient in-pipeline DIMC execution alongside standard vector units. Experimental results on ResNet-50 show peak DIMC performance of up to 137 GOPS with over 200× raw speedup and more than 50× area-normalized speedup compared to a baseline RVV core, validating the architecture under realistic edge constraints. The work demonstrates the potential of scalable, filter-friendly DIMC acceleration for edge AI with minimal hardware overhead and lays the groundwork for multi-tile scaling and broader workload support.

Abstract

Expanding Deep Learning applications toward edge computing demands architectures capable of delivering high computational performance and efficiency while adhering to tight power and memory constraints. Digital In-Memory Computing (DIMC) addresses this need by moving part of the computation directly within memory arrays, significantly reducing data movement and improving energy efficiency. This paper introduces a novel architecture that extends the Vector RISC-V Instruction Set Architecture (ISA) to integrate a tightly coupled DIMC unit directly into the execution stage of the pipeline, to accelerate Deep Learning inference at the edge. Specifically, the proposed approach adds four custom instructions dedicated to data loading, computation, and write-back, enabling flexible and optimal control of the inference execution on the target architecture. Experimental results demonstrate high utilization of the DIMC tile in Vector RISC-V and sustained throughput across the ResNet-50 model, achieving a peak performance of 137 GOP/s. The proposed architecture achieves a speedup of 217x over the baseline core and 50x area-normalized speedup even when operating near the hardware resource limits. The experimental results confirm the high potential of the proposed architecture as a scalable and efficient solution to accelerate Deep Learning inference on the edge.

Figures (6)

• Figure 1: Architectural approaches to improve data locality: from von Neumann designs to heterogeneous systems with near-memory and in-memory compputing.
• Figure 2: Architecture of DIMC Tile (P sub-arrays, J rows each) as presented at ISSCC 2023 desoli_167_2023
• Figure 3: Architecture overview with the DIMC tile Integrated in the Vector Execution Stage as a Parallel Execution Lane
• Figure 4: Bit-level encoding of the proposed custom instructions for DIMC accelerator integration. Customized bit fields are highlighted in blue.
• Figure 5: GOPS Achieved per Layer in ResNet50