Multimodal Chip Physical Design Engineer Assistant

TL;DR

This work tackles the gap between prediction accuracy and interpretability in chip physical design by introducing a Multimodal Large Language Model Assistant (MLLMA) that jointly predicts congestion and offers actionable design guidance. The approach combines automated feature generation via a Genetic Instruct framework with interpretable preferences that map features to design changes, packaged as a Design Suggestion Deck. Evaluations on CircuitNet show state-of-the-art congestion prediction performance and improved explainability, supported by ablations and case studies demonstrating actionable guidance. The framework enables context-aware, design-level reasoning across multimodal inputs (images, text, and graphs), offering engineers tangible paths to reduce congestion and optimize layouts in real-world workflows.

Abstract

Modern chip physical design relies heavily on Electronic Design Automation (EDA) tools, which often struggle to provide interpretable feedback or actionable guidance for improving routing congestion. In this work, we introduce a Multimodal Large Language Model Assistant (MLLMA) that bridges this gap by not only predicting congestion but also delivering human-interpretable design suggestions. Our method combines automated feature generation through MLLM-guided genetic prompting with an interpretable preference learning framework that models congestion-relevant tradeoffs across visual, tabular, and textual inputs. We compile these insights into a "Design Suggestion Deck" that surfaces the most influential layout features and proposes targeted optimizations. Experiments on the CircuitNet benchmark demonstrate that our approach outperforms existing models on both accuracy and explainability. Additionally, our design suggestion guidance case study and qualitative analyses confirm that the learned preferences align with real-world design principles and are actionable for engineers. This work highlights the potential of MLLMs as interactive assistants for interpretable and context-aware physical design optimization.

Figures (12)

• Figure 1: Overview of the Genetic Instruct framework. (a) Spatial metrics are extracted from routability images using domain-specific rules. (b) A Random Forest ranks and selects key features. (c) MLLM agents iteratively generate and mutate features via code, enriching the feature pool to improve downstream performance.
• Figure 2: Feature Pool Examples
• Figure 3: Overview of our interpretable preference modeling framework. The model leverages interpretable features identified in Phase 1 as preference signals to guide congestion prediction and design suggestion generation. A multimodal architecture integrates layout images and task prompts, predicts feature values, learns objective weights via a gating mechanism, and generates both scalar congestion scores and spatial congestion maps. This enables transparent and actionable feedback for physical design refinement.
• Figure 4: Qualitative comparison of congestion prediction heatmaps between the GPDL baseline and our proposed multimodal model, MLLMA. Red bounding boxes highlight congestion hotspots where MLLMA more accurately captures the spatial patterns and severity of congestion compared to the ground truth. Black bounding boxes indicate regions where GPDL fails to localize congestion accurately.
• Figure 5: By following the top model-attributed features and adjusting actionable design parameters, the resulting design exhibits significantly lower predicted and actual congestion.

Theorems & Definitions (3)

• Definition 1: SSIM
• Definition 2: MSE, NRMSE, PeakNRMSE
• Definition 3: Correlation-Based Metrics: PLCC, SRCC, KRCC