Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Robustness and Generalization of ML-Based Congestion Predictors to Valid and Imperceptible Perturbations

Chester Holtz, Yucheng Wang, Chung-Kuan Cheng, Bill Lin

TL;DR

This work investigates robustness in the context of ML-based EDA tools -- particularly for congestion prediction and describes a simple technique to train predictors that improves robustness to these perturbations.

Abstract

There is substantial interest in the use of machine learning (ML)-based techniques throughout the electronic computer-aided design (CAD) flow, particularly methods based on deep learning. However, while deep learning methods have achieved state-of-the-art performance in several applications, recent work has demonstrated that neural networks are generally vulnerable to small, carefully chosen perturbations of their input (e.g. a single pixel change in an image). In this work, we investigate robustness in the context of ML-based EDA tools -- particularly for congestion prediction. As far as we are aware, we are the first to explore this concept in the context of ML-based EDA. We first describe a novel notion of imperceptibility designed specifically for VLSI layout problems defined on netlists and cell placements. Our definition of imperceptibility is characterized by a guarantee that a perturbation to a layout will not alter its global routing. We then demonstrate that state-of-the-art CNN and GNN-based congestion models exhibit brittleness to imperceptible perturbations. Namely, we show that when a small number of cells (e.g. 1%-5% of cells) have their positions shifted such that a measure of global congestion is guaranteed to remain unaffected (e.g. 1% of the design adversarially shifted by 0.001% of the layout space results in a predicted decrease in congestion of up to 90%, while no change in congestion is implied by the perturbation). In other words, the quality of a predictor can be made arbitrarily poor (i.e. can be made to predict that a design is "congestion-free") for an arbitrary input layout. Next, we describe a simple technique to train predictors that improves robustness to these perturbations. Our work indicates that CAD engineers should be cautious when integrating neural network-based mechanisms in EDA flows to ensure robust and high-quality results.

On Robustness and Generalization of ML-Based Congestion Predictors to Valid and Imperceptible Perturbations

TL;DR

This work investigates robustness in the context of ML-based EDA tools -- particularly for congestion prediction and describes a simple technique to train predictors that improves robustness to these perturbations.

Abstract

There is substantial interest in the use of machine learning (ML)-based techniques throughout the electronic computer-aided design (CAD) flow, particularly methods based on deep learning. However, while deep learning methods have achieved state-of-the-art performance in several applications, recent work has demonstrated that neural networks are generally vulnerable to small, carefully chosen perturbations of their input (e.g. a single pixel change in an image). In this work, we investigate robustness in the context of ML-based EDA tools -- particularly for congestion prediction. As far as we are aware, we are the first to explore this concept in the context of ML-based EDA. We first describe a novel notion of imperceptibility designed specifically for VLSI layout problems defined on netlists and cell placements. Our definition of imperceptibility is characterized by a guarantee that a perturbation to a layout will not alter its global routing. We then demonstrate that state-of-the-art CNN and GNN-based congestion models exhibit brittleness to imperceptible perturbations. Namely, we show that when a small number of cells (e.g. 1%-5% of cells) have their positions shifted such that a measure of global congestion is guaranteed to remain unaffected (e.g. 1% of the design adversarially shifted by 0.001% of the layout space results in a predicted decrease in congestion of up to 90%, while no change in congestion is implied by the perturbation). In other words, the quality of a predictor can be made arbitrarily poor (i.e. can be made to predict that a design is "congestion-free") for an arbitrary input layout. Next, we describe a simple technique to train predictors that improves robustness to these perturbations. Our work indicates that CAD engineers should be cautious when integrating neural network-based mechanisms in EDA flows to ensure robust and high-quality results.
Paper Structure (14 sections, 11 equations, 7 figures, 2 tables)

This paper contains 14 sections, 11 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Preliminaries and Related Work
  4. Global routing
  5. RUDY
  6. Machine learning and EDA
  7. Machine learning and robustness
  8. A novel notion of imperceptibility for congestion prediction
  9. Experiments
  10. Experimental setup
  11. Algorithm parameters
  12. On the robustness of congestion predictors
  13. Improving robustness of congestion predictors via momentum-based PGD
  14. Conclusion and Future Work

Figures (7)

  • Figure 1: Notation
  • Figure 2: General illustration of effect of imperceptible perturbations on EDA predictions (a.): Vanilla prediction framework. The netlist-graph and cell attributes (i.e. positions) are used to make predictions (e.g. DRC locations or congestion hotspots) via the neural predictor. (b.): The attributes of a subset of nodes are perturbed: $x' = x + \delta_x$. The predictor is vulnerable (i.e. can be made to predict that a design is congestion-free)— even when both $\delta_x$ and the number of perturbed nodes are small.
  • Figure 3: Local constraints for each movable cell (highlighted in red) ensures cells do not move G-Cells.
  • Figure 4: Performance metrics associated with a vanilla network evaluated on unperturbed and perturbed layouts. (\ref{['fig:nrms']}) Distribution shift in NRMS. (\ref{['fig:ssim']}) Distribution shift in SSIM. congestion characterize a good predictor as achieving $NRMS < 0.2$ and $SSIM > 0.8$. Using our method, we are able create valid inputs such that approximately $100\%$ of samples have $SSIM < 0.8$ and $60\%$ of samples have $NRMS > 0.15$. $100\%$ of samples satisfy one of the two conditions.
  • Figure 5: Mean relative unsupervised error of PGD over iterations. Shaded region denotes 1 standard deviation. Note the log-scale in blue implies convergence in error.
  • ...and 2 more figures

Theorems & Definitions (1)

  • definition 1: $\epsilon$-robust