Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Joint Learning Approach to Hardware Caching and Prefetching

Samuel Yuan, Divyanshu Saxena, Jiayi Chen, Nihal Sharma, Aditya Akella

TL;DR

The paper tackles suboptimal performance arising when hardware caching policies for replacement and prefetching are trained in isolation. It introduces joint learning with shared representations, offering a joint encoder approach and a contrastive learning variant to coordinate the two policies. Preliminary experiments on ChampSim with SPEC CPU traces show consistent accuracy improvements of about 1–1.3×, with some traces nearing perfect eviction decisions. This work establishes a practical path toward synchronizing interdependent hardware policies and outlines concrete directions for extending joint learning to other domains and hardware constraints.

Abstract

Several learned policies have been proposed to replace heuristics for scheduling, caching, and other system components in modern systems. By leveraging diverse features, learning from historical trends, and predicting future behaviors, such models promise to keep pace with ever-increasing workload dynamism and continuous hardware evolution. However, policies trained in isolation may still achieve suboptimal performance when placed together. In this paper, we inspect one such instance in the domain of hardware caching -- for the policies of cache replacement and prefetching. We argue that these two policies are bidirectionally interdependent and make the case for training the two jointly. We propose a joint learning approach based on developing shared representations for the features used by the two policies. We present two approaches to develop these shared representations, one based on a joint encoder and another based on contrastive learning of the embeddings, and demonstrate promising preliminary results for both of these. Finally, we lay down an agenda for future research in this direction.

A Joint Learning Approach to Hardware Caching and Prefetching

TL;DR

The paper tackles suboptimal performance arising when hardware caching policies for replacement and prefetching are trained in isolation. It introduces joint learning with shared representations, offering a joint encoder approach and a contrastive learning variant to coordinate the two policies. Preliminary experiments on ChampSim with SPEC CPU traces show consistent accuracy improvements of about 1–1.3×, with some traces nearing perfect eviction decisions. This work establishes a practical path toward synchronizing interdependent hardware policies and outlines concrete directions for extending joint learning to other domains and hardware constraints.

Abstract

Several learned policies have been proposed to replace heuristics for scheduling, caching, and other system components in modern systems. By leveraging diverse features, learning from historical trends, and predicting future behaviors, such models promise to keep pace with ever-increasing workload dynamism and continuous hardware evolution. However, policies trained in isolation may still achieve suboptimal performance when placed together. In this paper, we inspect one such instance in the domain of hardware caching -- for the policies of cache replacement and prefetching. We argue that these two policies are bidirectionally interdependent and make the case for training the two jointly. We propose a joint learning approach based on developing shared representations for the features used by the two policies. We present two approaches to develop these shared representations, one based on a joint encoder and another based on contrastive learning of the embeddings, and demonstrate promising preliminary results for both of these. Finally, we lay down an agenda for future research in this direction.

Paper Structure

This paper contains 10 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Joint Learning
  4. Shared representations via joint encoder.
  5. Aligning representations via contrastive learning.
  6. On the pros and cons of each approach.
  7. Preliminary Results and Future Directions
  8. Training Setup
  9. Results
  10. Future Directions

Figures (1)

  • Figure 1: Logical diagrams for the model architectures for state-of-the-art learned replacement and prefetching policies (a), versus our proposed joint learned models: (b) using a joint encoder, and (c) using contrastive learning.