Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM4GNAS: A Large Language Model Based Toolkit for Graph Neural Architecture Search

Yang Gao, Hong Yang, Yizhi Chen, Junxian Wu, Peng Zhang, Haishuai Wang

TL;DR

LLM4GNAS tackles the adaptability bottleneck in Graph Neural Architecture Search by introducing an LLM-based toolkit that augments node features, guides architecture search, and optimizes hyperparameters via prompts. The framework uses an LLM as controller to iterative perform LLM-enhanced Node Augmentation, GNAS, and Hyperparameter Optimization, with prompt engineering enabling easy transfer to new search spaces. Empirical results on both homogeneous and heterogeneous graphs show competitive performance and improved search efficiency against established GNAS baselines, with ablations validating the contributions of node augmentation and LLM choice. This work demonstrates a scalable, extensible path toward prompt-driven neural architecture search for graph tasks and lays groundwork for graph foundation-models.

Abstract

Graph Neural Architecture Search (GNAS) facilitates the automatic design of Graph Neural Networks (GNNs) tailored to specific downstream graph learning tasks. However, existing GNAS approaches often require manual adaptation to new graph search spaces, necessitating substantial code optimization and domain-specific knowledge. To address this challenge, we present LLM4GNAS, a toolkit for GNAS that leverages the generative capabilities of Large Language Models (LLMs). LLM4GNAS includes an algorithm library for graph neural architecture search algorithms based on LLMs, enabling the adaptation of GNAS methods to new search spaces through the modification of LLM prompts. This approach reduces the need for manual intervention in algorithm adaptation and code modification. The LLM4GNAS toolkit is extensible and robust, incorporating LLM-enhanced graph feature engineering, LLM-enhanced graph neural architecture search, and LLM-enhanced hyperparameter optimization. Experimental results indicate that LLM4GNAS outperforms existing GNAS methods on tasks involving both homogeneous and heterogeneous graphs.

LLM4GNAS: A Large Language Model Based Toolkit for Graph Neural Architecture Search

TL;DR

LLM4GNAS tackles the adaptability bottleneck in Graph Neural Architecture Search by introducing an LLM-based toolkit that augments node features, guides architecture search, and optimizes hyperparameters via prompts. The framework uses an LLM as controller to iterative perform LLM-enhanced Node Augmentation, GNAS, and Hyperparameter Optimization, with prompt engineering enabling easy transfer to new search spaces. Empirical results on both homogeneous and heterogeneous graphs show competitive performance and improved search efficiency against established GNAS baselines, with ablations validating the contributions of node augmentation and LLM choice. This work demonstrates a scalable, extensible path toward prompt-driven neural architecture search for graph tasks and lays groundwork for graph foundation-models.

Abstract

Graph Neural Architecture Search (GNAS) facilitates the automatic design of Graph Neural Networks (GNNs) tailored to specific downstream graph learning tasks. However, existing GNAS approaches often require manual adaptation to new graph search spaces, necessitating substantial code optimization and domain-specific knowledge. To address this challenge, we present LLM4GNAS, a toolkit for GNAS that leverages the generative capabilities of Large Language Models (LLMs). LLM4GNAS includes an algorithm library for graph neural architecture search algorithms based on LLMs, enabling the adaptation of GNAS methods to new search spaces through the modification of LLM prompts. This approach reduces the need for manual intervention in algorithm adaptation and code modification. The LLM4GNAS toolkit is extensible and robust, incorporating LLM-enhanced graph feature engineering, LLM-enhanced graph neural architecture search, and LLM-enhanced hyperparameter optimization. Experimental results indicate that LLM4GNAS outperforms existing GNAS methods on tasks involving both homogeneous and heterogeneous graphs.

Paper Structure

This paper contains 26 sections, 1 equation, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. LLM4GNAS Framework
  3. LLM as Controller
  4. LLM-enhanced Node Augmentation
  5. LLM-enhanced GNAS
  6. LLM-enhanced Hyperparameter Optimization
  7. Interfaces of LLM4GNAS
  8. GNAS Example with LLM4GNAS
  9. GNAS Example with New Search Space
  10. Experiments
  11. Experiment Setup
  12. Baselines
  13. Hyperparameters
  14. Experimental setting
  15. Experimental environment
  16. ...and 11 more sections

Figures (3)

  • Figure 1: The overall framework of LLM4GNAS. LLM4GNAS leverages Large Language Models (LLMs) as the controller to automatically design Graph Neural Networks (GNNs) for downstream graph learning tasks. The framework consists of three components, i.e., LLM-based Feature Augmentation, Graph Neural Architecture Search, and Hyperparameter Optimization. Domain knowledge can be injected by prompts to guide the iterations towards the best architectures.
  • Figure 2: Variation in test accuracy of the best GNNs found at each iteration during the architecture search process for LLM4GNAS, GNAS-LLM, and AutoGEL on homogeneous graphs.
  • Figure 3: Iterations of the best GNN architectures generated by LLM4GNAS on homogeneous graphs. Colored blocks indicate modifications from the previous iteration.