HH-PIM: Dynamic Optimization of Power and Performance with Heterogeneous-Hybrid PIM for Edge AI Devices

TL;DR

The paper tackles the mismatch between fixed compute resources and dynamic inference workloads on edge devices by introducing HH-PIM. It presents a heterogeneous architecture with HP and LP PIM modules, each using a MRAM-SRAM hybrid memory, augmented by a dynamic data placement DP-based optimization to minimize energy under latency constraints. The authors validate the design with RTL-level modeling, FPGA prototyping, and 45 nm memory-power simulations across TinyML benchmarks, reporting up to 60.43% average energy savings over conventional PIMs. These results demonstrate that HH-PIM can adapt to real-time workload variations while meeting latency requirements, making it a viable approach for energy-efficient edge AI.

Abstract

Processing-in-Memory (PIM) architectures offer promising solutions for efficiently handling AI applications in energy-constrained edge environments. While traditional PIM designs enhance performance and energy efficiency by reducing data movement between memory and processing units, they are limited in edge devices due to continuous power demands and the storage requirements of large neural network weights in SRAM and DRAM. Hybrid PIM architectures, incorporating non-volatile memories like MRAM and ReRAM, mitigate these limitations but struggle with a mismatch between fixed computing resources and dynamically changing inference workloads. To address these challenges, this study introduces a Heterogeneous-Hybrid PIM (HH-PIM) architecture, comprising high-performance MRAM-SRAM PIM modules and low-power MRAM-SRAM PIM modules. We further propose a data placement optimization algorithm that dynamically allocates data based on computational demand, maximizing energy efficiency. FPGA prototyping and power simulations with processors featuring HH-PIM and other PIM types demonstrate that the proposed HH-PIM achieves up to $60.43$ percent average energy savings over conventional PIMs while meeting application latency requirements. These results confirm the suitability of HH-PIM for adaptive, energy-efficient AI processing in edge devices.

Figures (6)

• Figure 1: Block diagram of the proposed HH-PIM architecture.
• Figure 2: Block diagram of the HP-PIM Controller architecture.
• Figure 3: Design and FPGA prototyping of the HH-PIM processor.
• Figure 4: Various workload scenarios of the AI benchmark app.
• Figure 5: Energy savings of the HH-PIM over the Baseline-, Heterogeneous-, and Hybrid-PIM across benchmark scenarios.