Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Benchmarking End-To-End Performance of AI-Based Chip Placement Algorithms

Zhihai Wang, Zijie Geng, Zhaojie Tu, Jie Wang, Yuxi Qian, Zhexuan Xu, Ziyan Liu, Siyuan Xu, Zhentao Tang, Shixiong Kai, Mingxuan Yuan, Jianye Hao, Bin Li, Yongdong Zhang, Feng Wu

TL;DR

This work tackles the misalignment between surrogate metrics and end-to-end PPA in AI-based chip placement by introducing ChiPBench, an open, reproducible benchmark that delivers end-to-end evaluation over 20 diverse circuits using the full OpenROAD-based EDA workflow. It categorizes placement methods into Black-Box, Analytical, and RL families and evaluates six state-of-the-art AI-based algorithms, revealing that improvements on intermediate metrics do not reliably translate to final PPA gains. The key contributions are the open dataset with full design kits, the end-to-end evaluation framework, and the systematic demonstration that current intermediate metrics poorly predict final PPA, underscoring the need for end-to-end optimization targets. This benchmark has practical impact by providing a realistic, unified platform for researchers and industry to assess whether AI-driven placement methods deliver tangible PPA improvements in real designs.

Abstract

The increasing complexity of modern very-large-scale integration (VLSI) design highlights the significance of Electronic Design Automation (EDA) technologies. Chip placement is a critical step in the EDA workflow, which positions chip modules on the canvas with the goal of optimizing performance, power, and area (PPA) metrics of final chip designs. Recent advances have demonstrated the great potential of AI-based algorithms in enhancing chip placement. However, due to the lengthy workflow of chip design, the evaluations of these algorithms often focus on intermediate surrogate metrics, which are easy to compute but frequently reveal a substantial misalignment with the end-to-end performance (i.e., the final design PPA). To address this challenge, we introduce ChiPBench, which can effectively facilitate research in chip placement within the AI community. ChiPBench is a comprehensive benchmark specifically designed to evaluate the effectiveness of existing AI-based chip placement algorithms in improving final design PPA metrics. Specifically, we have gathered 20 circuits from various domains (e.g., CPU, GPU, and microcontrollers). These designs are compiled by executing the workflow from the verilog source code, which preserves necessary physical implementation kits, enabling evaluations for the placement algorithms on their impacts on the final design PPA. We executed six state-of-the-art AI-based chip placement algorithms on these designs and plugged the results of each single-point algorithm into the physical implementation workflow to obtain the final PPA results. Experimental results show that even if intermediate metric of a single-point algorithm is dominant, while the final PPA results are unsatisfactory. We believe that our benchmark will serve as an effective evaluation framework to bridge the gap between academia and industry.

Benchmarking End-To-End Performance of AI-Based Chip Placement Algorithms

TL;DR

This work tackles the misalignment between surrogate metrics and end-to-end PPA in AI-based chip placement by introducing ChiPBench, an open, reproducible benchmark that delivers end-to-end evaluation over 20 diverse circuits using the full OpenROAD-based EDA workflow. It categorizes placement methods into Black-Box, Analytical, and RL families and evaluates six state-of-the-art AI-based algorithms, revealing that improvements on intermediate metrics do not reliably translate to final PPA gains. The key contributions are the open dataset with full design kits, the end-to-end evaluation framework, and the systematic demonstration that current intermediate metrics poorly predict final PPA, underscoring the need for end-to-end optimization targets. This benchmark has practical impact by providing a realistic, unified platform for researchers and industry to assess whether AI-driven placement methods deliver tangible PPA improvements in real designs.

Abstract

The increasing complexity of modern very-large-scale integration (VLSI) design highlights the significance of Electronic Design Automation (EDA) technologies. Chip placement is a critical step in the EDA workflow, which positions chip modules on the canvas with the goal of optimizing performance, power, and area (PPA) metrics of final chip designs. Recent advances have demonstrated the great potential of AI-based algorithms in enhancing chip placement. However, due to the lengthy workflow of chip design, the evaluations of these algorithms often focus on intermediate surrogate metrics, which are easy to compute but frequently reveal a substantial misalignment with the end-to-end performance (i.e., the final design PPA). To address this challenge, we introduce ChiPBench, which can effectively facilitate research in chip placement within the AI community. ChiPBench is a comprehensive benchmark specifically designed to evaluate the effectiveness of existing AI-based chip placement algorithms in improving final design PPA metrics. Specifically, we have gathered 20 circuits from various domains (e.g., CPU, GPU, and microcontrollers). These designs are compiled by executing the workflow from the verilog source code, which preserves necessary physical implementation kits, enabling evaluations for the placement algorithms on their impacts on the final design PPA. We executed six state-of-the-art AI-based chip placement algorithms on these designs and plugged the results of each single-point algorithm into the physical implementation workflow to obtain the final PPA results. Experimental results show that even if intermediate metric of a single-point algorithm is dominant, while the final PPA results are unsatisfactory. We believe that our benchmark will serve as an effective evaluation framework to bridge the gap between academia and industry.
Paper Structure (30 sections, 1 equation, 5 figures, 27 tables)

This paper contains 30 sections, 1 equation, 5 figures, 27 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background on Electronic Design Automation
  4. Dataset
  5. Description of Designs
  6. Dataset Generation Pipeline
  7. Algorithms
  8. Black-Box-Optimization (BBO) Methods
  9. Analytical (Gradient-Based) Methods
  10. Reinforcement Learning (RL) Methods
  11. Evaluation
  12. Evaluation Metrics
  13. Final Design PPA Metrics
  14. Intermediate Metrics
  15. End-to-End Evaluation Workflow
  16. ...and 15 more sections

Figures (5)

  • Figure 1: Illustration of the modern chip design workflow.
  • Figure 2: Illustration of our end-to-end evaluation workflow.
  • Figure 3: Correlations Between Wirelength and TNS/WNS, In the visualizations, points that share the same color represent data from (a) same method or (b) same case, respectively.
  • Figure 4: Correlation Between Metrics
  • Figure 5: The images of the worst path for each method in ariane133.