BBOPlace-Bench: Benchmarking Black-Box Optimization for Chip Placement

TL;DR

BBOPlace-Bench introduces the first dedicated benchmark for evaluating black-box optimization in chip placement, decoupling problem formulations, optimization algorithms, and evaluation. It supports three formulations—Sequence Pair, Mask-Guided Optimization, and Hyperparameter Optimization—across SA, Vanilla-EA, ES, PSO, and BO, with MP-HPWL, GP-HPWL, and PPA as evaluation metrics. Experimental results on ISPD 2005 and ICCAD 2015 show that MGO and HPO formulations often outperform SP, with Vanilla-EA and PSO delivering strong performance across settings, and in some cases surpassing state-of-the-art analytical and RL methods. The work provides a flexible, real-world benchmark to accelerate BBO development for chip placement and suggests future directions in multi-objective, expensive, and high-dimensional optimization, as well as learning-enabled approaches.

Abstract

Chip placement is a vital stage in modern chip design as it has a substantial impact on the subsequent processes and the overall quality of the final chip. The use of black-box optimization (BBO) for chip placement has a history of several decades. However, early efforts were limited by immature problem formulations and inefficient algorithm designs. Recent progress has shown the effectiveness and efficiency of BBO for chip placement, proving its potential to achieve state-of-the-art results. Despite these advancements, the field lacks a unified, BBO-specific benchmark for thoroughly assessing various problem formulations and BBO algorithms. To fill this gap, we propose BBOPlace-Bench, the first benchmark designed specifically for evaluating and developing BBO algorithms for chip placement tasks. It integrates three problem formulations of BBO for chip placement, and offers a modular, decoupled, and flexible framework that enables users to seamlessly implement, test, and compare their own algorithms. BBOPlace-Bench integrates a wide variety of existing BBO algorithms, including simulated annealing (SA), evolutionary algorithms (EAs), and Bayesian optimization (BO). Experimental results show that the problem formulations of mask-guided optimization and hyperparameter optimization exhibit superior performance than the sequence pair problem formulation, while EAs demonstrate better overall performance than SA and BO, especially in high-dimensional search spaces, and also achieve state-of-the-art performance compared to the mainstream chip placement methods. BBOPlace-Bench not only facilitates the development of efficient BBO-driven solutions for chip placement but also broadens the practical application scenarios (which are urgently needed) for the BBO community. The code of BBOPlace-Bench is available at https://github.com/lamda-bbo/BBOPlace-Bench.

Figures (4)

• Figure 1: Illustration of BBOPlace-Bench. It decouples three core components, i.e., problem formulation, optimization algorithm, and evaluation, making it easier to test different BBO algorithms under customized settings. Three problem formulations are: 1) Sequence Pair: Using two permutations to capture positional relationships of macros. 2) Mask-Guided Optimization: Representing macros by grid coordinates, and using wire-mask-guided decoding to adjust the positions of macros to the grids with minimal incremental HPWL. 3) Hyperparameter Optimization: Optimizing the advanced analytical placer DREAMPlace’s hyperparameters (e.g., learning rate and target density) over mixed (discrete and continuous) search spaces, where the running of each configured DREAMPlace corresponds to the procedure of solution decoding, which generates a placement result of macros. Evaluation metrics include: MP-HPWL (fast proxy metric for wirelength of macros), GP-HPWL (accurate but costly metric for wirelength of macros and standard cells), and PPA (industrial chip metrics obtained via commercial tools like Cadence Innovus, a premier chip placement tool from the leading EDA provider Cadence Design Systems, Inc.).
• Figure 2: Example illustration of calculating HPWL. There are four modules $\{m_i\}_{i=1}^4$ on the chip canvas, where $P_{(i,j)}$ denotes the $j$-th pin of module $m_i$. There are two nets based on these modules, where net $e_1$ (colored in green) connects modules $m_1$, $m_2$ and $m_3$ using wires through pins $P_{(1,1)}$, $P_{(2,1)}$ and $P_{(3,2)}$, and net $e_2$ (colored in purple) connects modules $m_2$, $m_3$ and $m_4$ using wires through pins $P_{(2,2)}$, $P_{(3,1)}$ and $P_{(4,1)}$. $\mathrm{HPWL}_e$ is calculated as the half perimeter of the rectangle that encloses all the pins in net $e$; thus, the $\mathrm{HPWL}$ value of the current placement result is $w_1+h_1+w_2+h_2=4+5+5+5=19$.
• Figure 3: GP-HPWL vs. number of evaluations of different methods on ICCAD 2015.
• Figure 4: Placement layouts and congestions of different methods on the chip case superblue7 of ICCAD 2015. The congestion results are obtained by the commercial tool Cadence Innovus, where the red points indicate the congestion critical regions.