A Distributed Emulation Environment for In-Memory Computing Systems

TL;DR

The paper addresses the need for a real-time, configurable emulation platform to accelerate the development of in-memory computing (IMC) hardware for AI workloads at the edge. It presents the In-Memory Computing Emulator (IMCE), a modular system with analog and digital processing units (AnPU/DiPU), a front end, and a central data analytics server, complemented by a comprehensive software stack (HTT/HCT, Diolkos) that maps ONNX models to the emulator via im2col-based Conv2D and fused operations. The work provides detailed hardware architectures, a scalable software toolchain, and performance benchmarks on ResNet8/ResNet18 and YOLOv8n, showing high accuracy comparable to ONNX Runtime and practical throughput on a multi-board FPGA setup. Overall, IMCE enables rapid prototyping, detailed system-level analysis, microcode testing, and architectural exploration prior to silicon, offering significant practical impact for edge IMC hardware development.

Abstract

In-memory computing technology is used extensively in artificial intelligence devices due to lower power consumption and fast calculation of matrix-based functions. The development of such a device and its integration in a system takes a significant amount of time and requires the use of a real-time emulation environment, where various system aspects are analyzed, microcode is tested, and applications are deployed, even before the real chip is available. In this work, we present the architecture, the software development tools, and experimental results of a distributed and expandable emulation system for rapid prototyping of integrated circuits based on in-memory computing technologies. Presented experimental results demonstrate the usefulness of the proposed emulator.

Figures (5)

• Figure 1: The IMCE Architecture
• Figure 2: The IMCE-PU Architecture
• Figure 3: The IMCE with 4 NPU clusters
• Figure 4: From AI models to IMCE
• Figure 5: The ONNX to IMCE software tools