PIM-MMU: A Memory Management Unit for Accelerating Data Transfers in Commercial PIM Systems

TL;DR

PIM-MMU synergistically combines a hardware-based data copy engine, a PIM-optimized memory scheduler, and a heterogeneity-aware memory mapping function, the utilization of which is supported by the PIM-MMU software stack, significantly improving the efficiency of DRAM↔PIM data transfers.

Abstract

Processing-in-memory (PIM) has emerged as a promising solution for accelerating memory-intensive workloads as they provide high memory bandwidth to the processing units. This approach has drawn attention not only from the academic community but also from the industry, leading to the development of real-world commercial PIM devices. In this work, we first conduct an in-depth characterization on UPMEM's general purpose PIM system and analyze the bottlenecks caused by the data transfers across the DRAM and PIM address space. Our characterization study reveals several critical challenges associated with DRAM to/from PIM data transfers in memory bus integrated PIM systems, for instance, its high CPU core utilization, high power consumption, and low read/write throughput for both DRAM and PIM. Driven by our key findings, we introduce the PIM-MMU architecture which is a hardware/software codesign that enables energy-efficient DRAM to/from PIM transfers for PIM systems. PIM-MMU synergistically combines a hardwarebased data copy engine, a PIM-optimized memory scheduler, and a heterogeneity-aware memory mapping function, the utilization of which is supported by our PIM-MMU software stack, significantly improving the efficiency of DRAM to/from PIM data transfers. Experimental results show that PIM-MMU improves the DRAM to/from PIM data transfer throughput by an average 4.1x and enhances its energy-efficiency by 4.1x, leading to a 2.2x end-to-end speedup for real-world PIM workloads.

Figures (16)

• Figure 1: (a) Intel Xeon CPU server's memory system topology. (b)-(d) show the BIOS configuration related to different memory mapping functions and how they translate into exploiting MLP. While the BIOS configuration also supports N-way NUMA/rank/bank-level interleaving, we omit discussing them for brevity.
• Figure 2: Example showing how current PIM systems manage its DRAM and PIM physical address space.
• Figure 3: Bytes that constitute a given data word ('D','A','T','A','W','O','R','D') is colored identically. (a) Chip interleaving in a conventional DIMM-based memory system and (b) why UPMEM-PIM requires a transpose operation to be applied to the copied data beforehand to localize them within a single chip.
• Figure 4: The fraction of active CPU cores (left axis) and system power consumption (right axis) during (a) DRAM$\rightarrow$PIM and (b) PIM$\rightarrow$DRAM data transfer. System power consumption is measured using Intel's Performance Counter Monitor (PCM) intel_pcm.
• Figure 5: (a) MLP-optimized, hardware-based data transfer that maximally utilizes memory bandwidth. (b) Software-based data transfer using a single PIM thread which transfers data to its designated memory bank, leading to underutilization of memory bandwidth. Such limitation is better addressed in (c) which utilizes multiple concurrent PIM threads that target different channels/banks for data transfers, achieving higher memory throughput. In the pseudo-code in (b-c), the role of PIMcpy is conceptually identical to UPMEM-PIM's dpu_push_xfer or CUDA's cudaMemcpy APIs.