GraphEdit: Large Language Models for Graph Structure Learning

TL;DR

GraphEdit tackles noisy and incomplete graph structures by leveraging instruction-tuned LLMs to reason about node relationships and augment edges. It introduces a lightweight edge predictor to propose candidate edges, then uses LLM-based reasoning to refine the adjacency from the original graph via $A' = \text{EdgePredictor}(\mathcal{H}_n) + \mathcal{A}$ and $\hat{\mathcal{A}} = \text{LLM}(\text{Prompt}(A'))$. The approach integrates textual node content $t_n$ and prompt-based reasoning to denoise and reveal implicit node dependencies without relying solely on explicit supervision. Evaluations on Cora, Citeseer, and PubMed show GraphEdit achieving state-of-the-art accuracy and robustness to noise, with ablations validating the roles of instruction-tuning, edge prediction, and downstream GNN encoding. The work highlights the practical impact of combining LLM reasoning with graph refinement to improve downstream tasks and graph representations.

Abstract

Graph Structure Learning (GSL) focuses on capturing intrinsic dependencies and interactions among nodes in graph-structured data by generating novel graph structures. Graph Neural Networks (GNNs) have emerged as promising GSL solutions, utilizing recursive message passing to encode node-wise inter-dependencies. However, many existing GSL methods heavily depend on explicit graph structural information as supervision signals, leaving them susceptible to challenges such as data noise and sparsity. In this work, we propose GraphEdit, an approach that leverages large language models (LLMs) to learn complex node relationships in graph-structured data. By enhancing the reasoning capabilities of LLMs through instruction-tuning over graph structures, we aim to overcome the limitations associated with explicit graph structural information and enhance the reliability of graph structure learning. Our approach not only effectively denoises noisy connections but also identifies node-wise dependencies from a global perspective, providing a comprehensive understanding of the graph structure. We conduct extensive experiments on multiple benchmark datasets to demonstrate the effectiveness and robustness of GraphEdit across various settings. We have made our model implementation available at: https://github.com/HKUDS/GraphEdit.

Figures (3)

• Figure 1: The model architecture of our proposed GraphEdit framework for graph structure learning.
• Figure 2: Impact study of edge candidate selection.
• Figure 3: Visual analysis with random sampled 20 nodes and their 1-hot neighbors on the PubMed dataset.