Pickle Prefetcher: Programmable and Scalable Last-Level Cache Prefetcher

TL;DR

The paper tackles the challenge of irregular memory access patterns, particularly in graph workloads, by introducing the Pickle Prefetcher, a programmable LLC prefetcher that offloads prefetch logic to software. It replaces hardware prediction with a software-defined PicklePG, a dedicated PickleMMU, and a private Pickle Cache, enabling flexible, workload-specific prefetch strategies via an MMIO-based interface. The approach is evaluated in gem5 with BFS on GAPBS graphs, showing substantial speedups (up to 1.74x with private prefetchers) and improved timeliness when using prefetch drop thresholds; it also demonstrates increased LLC utilization and manageable memory-bandwidth trade-offs. The work argues for a scalable, adaptable hardware/software co-design that leverages software programmability to respond to evolving workloads without ISA changes, offering practical benefits for irregular-memory workloads and graph analytics.

Abstract

Modern high-performance architectures employ large last-level caches (LLCs). While large LLCs can reduce average memory access latency for workloads with a high degree of locality, they can also increase latency for workloads with irregular memory access patterns. Prefetchers are widely used to reduce memory latency by prefetching data into the cache hierarchy before it is accessed by the core. However, existing prediction-based prefetchers often struggle with irregular memory access patterns, which are especially prevalent in modern applications. This paper introduces the Pickle Prefetcher, a programmable and scalable LLC prefetcher designed to handle independent irregular memory access patterns effectively. Instead of relying on static heuristics or complex prediction algorithms, Pickle Prefetcher allows software to define its own prefetching strategies using a simple programming interface without expanding the instruction set architecture (ISA). By trading the logic complexity of hardware prediction for software programmability, Pickle Prefetcher can adapt to a wide range of access patterns without requiring extensive hardware resources for prediction. This allows the prefetcher to dedicate its resources to scheduling and issuing timely prefetch requests. Graph applications are an example where the memory access pattern is irregular but easily predictable by software. Through extensive evaluations of the Pickle Prefetcher on gem5 full-system simulations, we demonstrate tha Pickle Prefetcher significantly outperforms traditional prefetching techniques. Our results show that Pickle Prefetcher achieves speedups of up to 1.74x on the GAPBS breadth-first search (BFS) implementation over a baseline system. When combined with private cache prefetchers, Pickle Prefetcher provides up to a 1.40x speedup over systems using only private cache prefetchers.

Figures (13)

• Figure 1: An example of memory access pattern of the Breadth-first Search algorithm running on two cores. Each core may progress at a different pace, as a core can be scheduled out at any time. Here, the prefetch distance is set to 4, meaning the prefetcher fetches the node that is 4 positions ahead in the work_queue along with that node's neighbors. With a prefetch distance of 4, core 1 cannot issue a prefetch because it has only one unit of work remaining, while core 2 is able to issue a prefetch.
• Figure 2: The architecture of the Pickle Prefetcher with data paths. The prefetcher and the data paths are described in Section \ref{['sec:pickle_architecture']}.
• Figure 3: Data paths of cache accesses involving the Pickle Prefetcher. The data paths are described in Section \ref{['sec:pickle_data_path']}. The NoC is described in Section \ref{['sec:noc_configuration']}.
• Figure 4: Data paths for sending a prefetch hint from the core to the prefetcher. At the beginning of the program, a UC page is created (), mapping a 4KiB page to the prefetcher I/O physical address. Prefetch hints are sent from the core to the prefetcher via uncacheable store requests (and ). The virtual address of the prefetch determines the prefetch kernel, as shown in (and ). More details are provided in Section \ref{['section:offloading']}.
• Figure 5: Speedup of various prefetcher configurations over the baseline system. The graphs are sorted with increasing sizes from left to right. The result is discussed in Section \ref{['sec:speedups_prefetcher']}.