A Benchmark on Directed Graph Representation Learning in Hardware Designs

TL;DR

This work addresses the scarcity of robust benchmarks for directed graph representation learning (DGRL) in hardware design by proposing a modular benchmark with five hardware datasets and thirteen tasks, spanning digital and analog domains. It systematically evaluates 21 DGRL configurations that combine various backbones, message-passing directions, transformer variants, and direction-aware positional encodings, introducing the stable edge PE (EPE) approach. Key findings show that bidirected (BI) message passing and stable positional encodings substantially boost performance, with BI-GPS-T+EPE and BI-GIN+EPE often achieving top results, though OOD generalization remains a major challenge. The work provides a practical toolbox and a “recipe” for applying DGRL to hardware data, enabling hardware and ML practitioners to evaluate and design effective directed-graph surrogates while highlighting the need for improved OOD strategies in hardware contexts.

Abstract

To keep pace with the rapid advancements in design complexity within modern computing systems, directed graph representation learning (DGRL) has become crucial, particularly for encoding circuit netlists, computational graphs, and developing surrogate models for hardware performance prediction. However, DGRL remains relatively unexplored, especially in the hardware domain, mainly due to the lack of comprehensive and user-friendly benchmarks. This study presents a novel benchmark comprising five hardware design datasets and 13 prediction tasks spanning various levels of circuit abstraction. We evaluate 21 DGRL models, employing diverse graph neural networks and graph transformers (GTs) as backbones, enhanced by positional encodings (PEs) tailored for directed graphs. Our results highlight that bidirected (BI) message passing neural networks (MPNNs) and robust PEs significantly enhance model performance. Notably, the top-performing models include PE-enhanced GTs interleaved with BI-MPNN layers and BI-Graph Isomorphism Network, both surpassing baselines across the 13 tasks. Additionally, our investigation into out-of-distribution (OOD) performance emphasizes the urgent need to improve OOD generalization in DGRL models. This benchmark, implemented with a modular codebase, streamlines the evaluation of DGRL models for both hardware and ML practitioners

Figures (9)

• Figure 1: The benchmark considers 21 combinations of message passing direction, GNN backbone, transformer selection and PE incorporation, covers 10 existing SOTA methods from graph ML community and discovers 2 novel top-performing models (see Table. \ref{['tab:method_eixsting_proposed']}).
• Figure 2: Illustration of the directed graph representation learning (DGRL) toolbox.
• Figure 3: An example C program from the CDFG dataset.
• Figure 4: Control dataflow graph of the example program in Figure \ref{['fig:hls_cdfg:a']}.
• Figure 5: 8-bit multiplier in AIG.