CMD: A Cache-assisted GPU Memory Deduplication Architecture

TL;DR

The paper tackles the GPU off-chip DRAM bottleneck by introducing CMD, a cache-assisted GPU memory deduplication architecture. CMD detects intra- and inter-line duplicates using a strong hash, manages duplicate blocks with on-chip metadata, accelerates reads via a cache-assisted path, and chips in a read-only FIFO to reduce repeated DRAM fetches. The approach yields substantial reductions in off-chip traffic (≈31%), energy (≈33%), and improves IPC by up to ≈38%, with especially large gains on memory-intensive workloads. Its modular design suggests strong practical impact for memory-bound GPU workloads and compatibility with complementary compression techniques.

Abstract

Massive off-chip accesses in GPUs are the main performance bottleneck, and we divided these accesses into three types: (1) Write, (2) Data-Read, and (3) Read-Only. Besides, We find that many writes are duplicate, and the duplication can be inter-dup and intra-dup. While inter-dup means different memory blocks are identical, and intra-dup means all the 4B elements in a line are the same. In this work, we propose a cache-assisted GPU memory deduplication architecture named CMD to reduce the off-chip accesses via utilizing the data duplication in GPU applications. CMD includes three key design contributions which aim to reduce the three kinds of accesses: (1) A novel GPU memory deduplication architecture that removes the inter-dup and inter-dup lines. As for the inter-dup detection, we reduce the extra read requests caused by the traditional read-verify hash process. Besides, we design several techniques to manage duplicate blocks. (2) We propose a cache-assisted read scheme to reduce the reads to duplicate data. When an L2 cache miss wants to read the duplicate block, if the reference block has been fetched to L2 and it is clean, we can copy it to the L2 missed block without accessing off-chip DRAM. As for the reads to intra-dup data, CMD uses the on-chip metadata cache to get the data. (3) When a cache line is evicted, the clean sectors in the line are invalidated while the dirty sectors are written back. However, most read-only victims are re-referenced from DRAM more than twice. Therefore, we add a full-associate FIFO to accommodate the read-only (it is also clean) victims to reduce the re-reference counts. Experiments show that CMD can decrease the off-chip accesses by 31.01%, reduce the energy by 32.78% and improve performance by 37.79%. Besides, CMD can improve the performance of memory-intensive workloads by 50.18%.

Figures (19)

• Figure 1: The memory architecture of GPU.
• Figure 2: The off-chip access ratio and requests breakdown.
• Figure 3: The duplication ratio of several benchmarks.
• Figure 4: The overview of our CMD architecture.
• Figure 5: Two methods of duplicate memory lines detection.