DEER: Deep Runahead for Instruction Prefetching on Modern Mobile Workloads

TL;DR

DEER tackles frontend stalls in modern mobile workloads by combining software-generated metadata with a hardware deep runahead unit, enabling long-horizon instruction prefetching without heavy on-chip storage. By representing execution paths as HyperBlocks and predicting the Most Likely Successor, DEER prefetches through L2 with a Return Address Stack guiding return-path analytics, while self-correcting as execution diverges. The approach yields up to 19.9% average reduction in L2 I-miss rate and up to 8% IPC/speedup, outperforming full-hardware record-and-replay and other co-designed schemes by up to 4× gains at two orders of magnitude lower on-chip storage. This SW/HW co-design, with metadata in DRAM and a compact hardware DRU, provides practical, scalable improvements for mobile frontends and paves the way for broader use of memory-resident metadata in prefetching.

Abstract

Mobile workloads incur heavy frontend stalls due to increasingly large code footprints as well as long repeat cycles. Existing instruction-prefetching techniques suffer from low coverage, poor timeliness, or high cost. We provide a SW/HW co-designed I-prefetcher; DEER uses profile analysis to extract metadata information that allow the hardware to prefetch the most likely future instruction cachelines, hundreds of instructions earlier. This profile analysis skips over loops and recursions to go deeper into the future, and uses a return-address stack on the hardware side to allow prefetch on the return-path from large call-stacks. The produced metadata table is put in DRAM, pointed to by an in-hardware register; the high depth of the lookahead allows to preload the metadata in time and thus nearly no on-chip metadata storage is needed. Gem5 evaluation on real-world modern mobile workloads shows up to 45% reduction in L2 instruction-miss rate (19.6% on average), resulting in up to 8% speedup (4.7% on average). These gains are up to 4X larger than full-hardware record-and-replay prefetchers, while needing two orders of magnitude smaller on-chip storage.

Figures (17)

• Figure 1: Repeat distance of PC instances; Mobile applications exhibit a far longer tail than SPEC workloads.
• Figure 2: IPC gains by an oracle N-instructions-ahead prefetcher for different prefetch distances.
• Figure 3: Box-and-whisker plot for path prediction accuracy of the MLS (most likely successor) scheme.
• Figure 4: DEER system overview: deep runahead with software-generated metadata.
• Figure 5: An example of HB metadata for dynamic as well as SSRA runahead modes for a runahead depth of 5 HBs.