Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Intelligent4DSE: Optimizing High-Level Synthesis Design Space Exploration with Graph Neural Networks and Large Language Models

Lei Xu, Shanshan Wang, Emmanuel Casseau, Chenglong Xiao

TL;DR

This work tackles high-level synthesis design space exploration (HLS DSE) by addressing two core challenges: accurate QoR prediction and efficient Pareto-front search. It introduces ECoGNNs, a task-adaptive graph neural network that mitigates over-smoothing and long-range dependencies, paired with LLMMH, a framework that uses in-context learning to guide meta-heuristic search (NSGA-II, SA, ACO) without fine-tuning. Across post-HLS and post-implementation QoR tasks, ECoGNNs achieve substantial prediction accuracy gains, while LLMMH produces superior Pareto fronts, achieving up to 87.47% ADRS improvements and notable reductions relative to state-of-the-art baselines. The results demonstrate a promising paradigm for combining adaptive GNNs with LLM-guided optimization to accelerate hardware design space exploration, with future work focusing on runtimes, prompt design, and broader applicability to DSE problems.

Abstract

High-Level Synthesis (HLS) Design Space Exploration (DSE) is essential for generating hardware designs that balance performance, power, and area (PPA). To optimize this process, existing works often employs message-passing neural networks (MPNNs) to predict quality of results (QoR). These predictors serve as evaluators in the DSE process, effectively bypassing the time-consuming estimations traditionally required by HLS tools. However, existing models based on MPNNs struggle with over-smoothing and limited expressiveness. Additionally, while meta-heuristic algorithms are widely used in DSE, they typically require extensive domain-specific knowledge to design operators and time-consuming tuning. To address these limitations, we propose ECoGNNs-LLMMHs, a framework that integrates graph neural networks with task-adaptive message passing and large language model-enhanced meta-heuristic algorithms. Compared with state-of-the-art works, ECoGNN exhibits lower prediction error in the post-HLS prediction task, with the error reduced by 57.27\%. For post-implementation prediction tasks, ECoGNN demonstrates the lowest prediction errors, with average reductions of 17.6\% for flip-flop (FF) usage, 33.7\% for critical path (CP) delay, 26.3\% for power consumption, 38.3\% for digital signal processor (DSP) utilization, and 40.8\% for BRAM usage. LLMMH variants can generate superior Pareto fronts compared to meta-heuristic algorithms in terms of average distance from the reference set (ADRS) with average improvements of 87.47\%, respectively. Compared with the SOTA DSE approaches GNN-DSE and IRONMAN-PRO, LLMMH can reduce the ADRS by 68.17\% and 63.07\% respectively.

Intelligent4DSE: Optimizing High-Level Synthesis Design Space Exploration with Graph Neural Networks and Large Language Models

TL;DR

This work tackles high-level synthesis design space exploration (HLS DSE) by addressing two core challenges: accurate QoR prediction and efficient Pareto-front search. It introduces ECoGNNs, a task-adaptive graph neural network that mitigates over-smoothing and long-range dependencies, paired with LLMMH, a framework that uses in-context learning to guide meta-heuristic search (NSGA-II, SA, ACO) without fine-tuning. Across post-HLS and post-implementation QoR tasks, ECoGNNs achieve substantial prediction accuracy gains, while LLMMH produces superior Pareto fronts, achieving up to 87.47% ADRS improvements and notable reductions relative to state-of-the-art baselines. The results demonstrate a promising paradigm for combining adaptive GNNs with LLM-guided optimization to accelerate hardware design space exploration, with future work focusing on runtimes, prompt design, and broader applicability to DSE problems.

Abstract

High-Level Synthesis (HLS) Design Space Exploration (DSE) is essential for generating hardware designs that balance performance, power, and area (PPA). To optimize this process, existing works often employs message-passing neural networks (MPNNs) to predict quality of results (QoR). These predictors serve as evaluators in the DSE process, effectively bypassing the time-consuming estimations traditionally required by HLS tools. However, existing models based on MPNNs struggle with over-smoothing and limited expressiveness. Additionally, while meta-heuristic algorithms are widely used in DSE, they typically require extensive domain-specific knowledge to design operators and time-consuming tuning. To address these limitations, we propose ECoGNNs-LLMMHs, a framework that integrates graph neural networks with task-adaptive message passing and large language model-enhanced meta-heuristic algorithms. Compared with state-of-the-art works, ECoGNN exhibits lower prediction error in the post-HLS prediction task, with the error reduced by 57.27\%. For post-implementation prediction tasks, ECoGNN demonstrates the lowest prediction errors, with average reductions of 17.6\% for flip-flop (FF) usage, 33.7\% for critical path (CP) delay, 26.3\% for power consumption, 38.3\% for digital signal processor (DSP) utilization, and 40.8\% for BRAM usage. LLMMH variants can generate superior Pareto fronts compared to meta-heuristic algorithms in terms of average distance from the reference set (ADRS) with average improvements of 87.47\%, respectively. Compared with the SOTA DSE approaches GNN-DSE and IRONMAN-PRO, LLMMH can reduce the ADRS by 68.17\% and 63.07\% respectively.
Paper Structure (15 sections, 14 equations, 6 figures, 11 tables, 3 algorithms)

This paper contains 15 sections, 14 equations, 6 figures, 11 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Preliminaries
  4. Meta-heuristic Algorithms for DSE
  5. In-context Learning
  6. Methodology
  7. Dataset Generation
  8. Enhanced Cooperative Graph Neural Networks
  9. Large Language Model aided Meta-Heuristic Algorithms
  10. Experiments
  11. Experimental Setup
  12. Evaluation of Post-HLS QoR Prediction Accuracy
  13. Evaluation of Post-Implementation QoR Prediction Accuracy
  14. Evaluation of Design Space Exploration
  15. Conclusion

Figures (6)

  • Figure 1: A comprehensive overview of the HLS DSE process uses the atax application from the PolyBench benchmark suite. The HLS tools can be replaced by predictive models.
  • Figure 2: The ECoGNNs-LLMMH framework integrates data generation (combining HLS reports and implementation reports), predictive model training via task-adaptive graph neural networks, and DSE using an LLM-enhanced evolutionary algorithm. The ECoGNNs can be tuned to predict post-HLS QoR metrics and post-implementation QoR metrics.
  • Figure 3: The input graph $H$ and the computational graphs $H_0$, $H_1$, $H_2$ in MPNN and ECoGNN, where the computational graphs serve as an abstract representation of the message-passing process, with arrow directions indicating the flow of information.
  • Figure 4: Schematic diagram of ECoGNN's internal structure.
  • Figure 5: The LLMMH framework fully leverages the in-context learning capability of LLMs. By constructing an in-context prompt, it guides LLMs to understand the DSE task and further assists metaheuristic algorithms in generating new solutions. Specifically, the in-context prompt comprises task descriptions, solution examples, and task instructions. This workflow consists of initialization, LLM-guided solution generation, and GNN-based evaluation. In this study, we present and evaluate three LLM-driven meta-heuristics: LLMGA, LLMSA, and LLMACO.
  • ...and 1 more figures