Agile TLB Prefetching and Prediction Replacement Policy

TL;DR

The paper addresses the high latency of TLB misses in paging-based memory systems and proposes a unified software-hardware solution that combines an Agile TLB Prefetcher (ATP) with Sampling-Based Free TLB Prefetching (SBFP) to exploit page-table locality and prefetch essential free PTEs during page walks. It further discusses predictive replacement policies, notably CHiRP, which uses control-flow history to detect dead blocks in the L2 TLB, complementing prefetching to reduce misses and stalls. This integrated approach aims to lower translation latency and energy by both reducing misses and improving replacement decisions. The work identifies future directions, including neural predictors (e.g., RNNs/LSTMs) and multi-headed architectures to enhance both SBFP and ATP, with potential specialization for L2 TLBs. Overall, the framework highlights a path toward more efficient virtual memory management in contemporary CPUs through coordinated prefetching and history-based prediction mechanisms.

Abstract

Virtual-to-physical address translation is a critical performance bottleneck in paging-based virtual memory systems. The Translation Lookaside Buffer (TLB) accelerates address translation by caching frequently accessed mappings, but TLB misses lead to costly page walks. Hardware and software techniques address this challenge. Hardware approaches enhance TLB reach through system-level support, while software optimizations include TLB prefetching, replacement policies, superpages, and page size adjustments. Prefetching Page Table Entries (PTEs) for future accesses reduces bottlenecks but may incur overhead from incorrect predictions. Integrating an Agile TLB Prefetcher (ATP) with SBFP optimizes performance by leveraging page table locality and dynamically identifying essential free PTEs during page walks. Predictive replacement policies further improve TLB performance. Traditional LRU replacement is limited to near-instant references, while advanced policies like SRRIP, GHRP, SHiP, SDBP, and CHiRP enhance performance by targeting specific inefficiencies. CHiRP, tailored for L2 TLBs, surpasses other policies by leveraging control flow history to detect dead blocks, utilizing L2 TLB entries for learning instead of sampling. These integrated techniques collectively address key challenges in virtual memory management.