Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Updated Assessment of Reinforcement Learning for Macro Placement

Chung-Kuan Cheng, Andrew B. Kahng, Sayak Kundu, Yucheng Wang, Zhiang Wang

TL;DR

An improved assessment of Google Brain's deep reinforcement learning approach to macro placement and its updated Circuit Training (CT) implementation is provided in GitHub and insights into reproducibility and reporting in the research literature are affords.

Abstract

We provide an improved assessment of Google Brain's deep reinforcement learning approach to macro placement and its updated Circuit Training (CT) implementation in GitHub. A stronger simulated annealing (SA) baseline leverages the "go-with-the-winners" metaheuristic and a multi-threading implementation. We develop and release new public benchmarks in sub-10nm technology: LEF/DEF for Google's 7nm TSMC Ariane protobuf and scaled variants, as well as testcases implemented in the open-source ASAP7 7nm research enablement. We evaluate from-scratch training and fine-tuning results for the latest "AlphaChip" release of Circuit Training, alongside multiple alternative macro placers. We also study the recently-published pre-training guidance in. A commercial place-and-route tool is used to provide "true reward" post-route power, performance and area metrics. All data, evaluation flows and related scripts are publicly available in the MacroPlacement GitHub repository. Our study affords insights into reproducibility and reporting in the research literature, and points out still-missing confirmations (e.g., of CT's scalability and pre-training methodology) that remain open questions for the research community.

An Updated Assessment of Reinforcement Learning for Macro Placement

TL;DR

An improved assessment of Google Brain's deep reinforcement learning approach to macro placement and its updated Circuit Training (CT) implementation is provided in GitHub and insights into reproducibility and reporting in the research literature are affords.

Abstract

We provide an improved assessment of Google Brain's deep reinforcement learning approach to macro placement and its updated Circuit Training (CT) implementation in GitHub. A stronger simulated annealing (SA) baseline leverages the "go-with-the-winners" metaheuristic and a multi-threading implementation. We develop and release new public benchmarks in sub-10nm technology: LEF/DEF for Google's 7nm TSMC Ariane protobuf and scaled variants, as well as testcases implemented in the open-source ASAP7 7nm research enablement. We evaluate from-scratch training and fine-tuning results for the latest "AlphaChip" release of Circuit Training, alongside multiple alternative macro placers. We also study the recently-published pre-training guidance in. A commercial place-and-route tool is used to provide "true reward" post-route power, performance and area metrics. All data, evaluation flows and related scripts are publicly available in the MacroPlacement GitHub repository. Our study affords insights into reproducibility and reporting in the research literature, and points out still-missing confirmations (e.g., of CT's scalability and pre-training methodology) that remain open questions for the research community.
Paper Structure (19 sections, 6 figures, 12 tables, 1 algorithm)

This paper contains 19 sections, 6 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Macro Placement Methods
  3. Replication of Circuit Training
  4. Discrepancies between CT and Nature
  5. Computing resources for Circuit Training
  6. A Stronger Annealing Baseline
  7. Experimental Setup
  8. Testcases and enablements
  9. Commercial evaluation flow
  10. Settings for Circuit Training
  11. Experiments and Results
  12. Comparison of CT with other macro placers
  13. Stability studies
  14. Ablation studies
  15. Studies of Pre-training
  16. ...and 4 more sections

Figures (6)

  • Figure 1: Train steps per second plot for our CT run (left) and the CT run available (right) in the CT repository for Ariane.
  • Figure 2: Evaluation flow for placements produced by different macro placers.
  • Figure 3: Macro placement solutions for the Ariane-ASAP7 design, produced by different macro placers.
  • Figure 4: Loss (left) and train steps per second (right) for converged (red) and diverged (blue) CT runs on Ariane-GF12. Both runs use the same machine, same environment, same netlist, and same (default) seed = 333.
  • Figure 5: The impact of pre-training versus training from scratch on performance (higher placement return on the y-axis is better) for (a) Ariane-ASAP7, (b) Ariane-GF12 and (c) Ariane-NG45. Here, the placement return is defined as the negative of the proxy cost.
  • ...and 1 more figures