PIM-malloc: A Fast and Scalable Dynamic Memory Allocator for Processing-In-Memory (PIM) Architectures

TL;DR

This work tackles the absence of dynamic memory allocation support in commodity PIM by conducting a design-space study of allocator metadata placement and management. It introduces PIM-malloc in two variants—software-only and hardware/software co-design—showing that software-only achieves $66\times$ faster allocations than a straw-man baseline, with an additional $31\%$ improvement from a per-PIM-core cache, and that the hardware-enhanced version further reduces backend latency on real workloads to reach $28\times$ throughput gains. The approach combines per-thread caches to service small allocations and a scaled-back buddy allocator for larger requests, and augments it with a hardware buddy cache and ISA extensions to accelerate metadata access. Evaluation on real UPMEM-PIM hardware and cycle-accurate simulation demonstrates strong performance, reduced DRAM transfers, and manageable hardware overhead, highlighting the feasibility of scalable dynamic allocators for processing-in-memory systems. Overall, PIM-malloc provides a practical pathway to use dynamic data structures and graph workloads on PIM, significantly advancing programmability and efficiency in PIM architectures.

Abstract

Dynamic memory allocation is essential in modern programming but remains under-supported in current PIM devices. In this work, we first conduct a design space exploration of PIM memory allocators, examining optimal metadata placement and management strategies. Building on these insights, we propose PIM-malloc, a fast and scalable allocator for real PIM hardware, improving allocation performance by $66\times$. We further enhance this design with a lightweight, per-PIM core hardware cache for dynamic allocation, achieving an additional $31\%$ performance gain. Finally, we demonstrate the effectiveness of PIM-malloc using a dynamic graph update workload, achieving a $28\times$ throughput increase.

Figures (16)

• Figure 1: UPMEM-PIM hardware system overview.
• Figure 2: An example workflow of buddy allocator.
• Figure 3: Illustration of (a) a dynamic graph update operation, (b) a comparison of how static and dynamic data structures are used to manage the graph update shown in (a), and (c) the performance slowdown (normalized to static/small) as the size of the pre-update graph is increased, from small to large, while the total number of newly added graph edges remains constant. All experiments are conducted on a real UPMEM-PIM device. The graph updates using dynamic data structures are implemented using our proposed PIM-malloc-SW APIs, discussed in detail in Section \ref{['sect:proposed_sw_only']}. The loc-gowalla cho2011friendship dataset is used to create the pre-update graph data.
• Figure 4: Pseudo-code of how the design strategies in Table \ref{['tab:design_space_exploration']} can be utilized to implement our straw-man buddy_alloc_PIM_DRAM design. Pseudo-code features three primary functions: pimMalloc() allocates memory within the heap space in PIM's DRAM banks; pimMemcpy() (host-side code only) handles DRAM$\leftrightarrow$PIM data transfer; pimLaunch() (host-side code only) instructs PIM cores to execute the buddy allocation algorithm on the PIM side.
• Figure 5: (a) Average PIM memory allocation latency when employing the four design strategies discussed in Table \ref{['tab:design_space_exploration']}. This experiment assumes a scenario where 1 to 512 PIM cores request an identical 128 memory allocations with the size of 32 bytes concurrently. (b) Latency breakdown with 512 PIM cores data point in (a).