Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Compiler Support for Speculation in Decoupled Access/Execute Architectures

Robert Szafarczyk, Syed Waqar Nabi, Wim Vanderbauwhede

TL;DR

This work tackles the loss of decoupling in decoupled access/execute (DAE) architectures caused by control dependencies on memory accesses. It introduces compiler based speculation that hoists memory requests in the address generation slice and poisons mis speculations in the compute slice, backed by proofs of sequential consistency. The method handles reducible control flow by leveraging topological order and a rigorous mapping between CFG edges and blocks, enabling DAE for irregular workloads across CPUs, GPUs, CGRAs, and HLS accelerators. Evaluation on HLS generated accelerators for graph and data analytics demonstrates average speedups around 1.9x (up to 3x) with minimal area overhead and no mis speculations, highlighting practical impact for accelerator design targeting irregular patterns.

Abstract

Irregular codes are bottlenecked by memory and communication latency. Decoupled access/execute (DAE) is a common technique to tackle this problem. It relies on the compiler to separate memory address generation from the rest of the program, however, such a separation is not always possible due to control and data dependencies between the access and execute slices, resulting in a loss of decoupling. In this paper, we present compiler support for speculation in DAE architectures that preserves decoupling in the face of control dependencies. We speculate memory requests in the access slice and poison mis-speculations in the execute slice without the need for replays or synchronization. Our transformation works on arbitrary, reducible control flow and is proven to preserve sequential consistency. We show that our approach applies to a wide range of architectural work on CPU/GPU prefetchers, CGRAs, and accelerators, enabling DAE on a wider range of codes than before.

Compiler Support for Speculation in Decoupled Access/Execute Architectures

TL;DR

This work tackles the loss of decoupling in decoupled access/execute (DAE) architectures caused by control dependencies on memory accesses. It introduces compiler based speculation that hoists memory requests in the address generation slice and poisons mis speculations in the compute slice, backed by proofs of sequential consistency. The method handles reducible control flow by leveraging topological order and a rigorous mapping between CFG edges and blocks, enabling DAE for irregular workloads across CPUs, GPUs, CGRAs, and HLS accelerators. Evaluation on HLS generated accelerators for graph and data analytics demonstrates average speedups around 1.9x (up to 3x) with minimal area overhead and no mis speculations, highlighting practical impact for accelerator design targeting irregular patterns.

Abstract

Irregular codes are bottlenecked by memory and communication latency. Decoupled access/execute (DAE) is a common technique to tackle this problem. It relies on the compiler to separate memory address generation from the rest of the program, however, such a separation is not always possible due to control and data dependencies between the access and execute slices, resulting in a loss of decoupling. In this paper, we present compiler support for speculation in DAE architectures that preserves decoupling in the face of control dependencies. We speculate memory requests in the access slice and poison mis-speculations in the execute slice without the need for replays or synchronization. Our transformation works on arbitrary, reducible control flow and is proven to preserve sequential consistency. We show that our approach applies to a wide range of architectural work on CPU/GPU prefetchers, CGRAs, and accelerators, enabling DAE on a wider range of codes than before.
Paper Structure (34 sections, 1 theorem, 7 figures, 2 tables, 3 algorithms)

This paper contains 34 sections, 1 theorem, 7 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Motivating Example
  3. Background
  4. Architectural Support
  5. Compiler Preliminaries
  6. Loss-of-Decoupling Analysis
  7. Compiler Support for Speculation
  8. Speculating Memory Requests
  9. Example of Hoisting
  10. Nested LoD Control Dependencies
  11. Why Topological Order in Algorithm \ref{['alg:speculation']}?
  12. Poisoning Mis-speculated Stores
  13. Example of Mapping Poison Stores to CFG Edges
  14. Mapping Poisoned Edges to Basic Blocks
  15. Example of Mapping Poison Edges to Blocks
  16. ...and 19 more sections

Key Result

Lemma 6.1

Given an ordered list of $n$ speculative store requests $L_a = \{a_0,\, a_1,\, ...,\, a_{n-1}\}$ made in the AGU loop CFG on some fixed iteration $k$, Algorithms alg:poison and alg:poison_edge transform the CU CFG such that every possible path through its loop CFG on iteration $k$ produces an ordere

Figures (7)

  • Figure 1: A decoupled access/execute architecture template.
  • Figure 2: Comparison of a decoupled and non-decoupled address generation. Non-decoupled address generation results in a later arrival of the store address, which stalls the RAW check for the next load, lowering load throughput.
  • Figure 3: Poisoning speculated stores immediately when they become unreachable results in an ordering mismatch between AGU store requests from and CU store values.
  • Figure 4: An example of introducing speculative memory requests in the AGU (§ \ref{['sec:Speculative_Memory_Requests']}); and poisoned stores in the CU (§ \ref{['sec:Poisoning_mis_speculated_Stores']}). Block 6 in subfigure (d) kills stores c, b, then uses the allocation for store d, and then kills store e.
  • Figure 5: Basic blocks with the same list of poison stores and the same immediate successor can be merged in the CU.
  • ...and 2 more figures

Theorems & Definitions (3)

  • Definition 4.1: LoD Data Dependency
  • Definition 4.2: LoD Control Dependency
  • Lemma 6.1: Sequential Consistency