Large Language Models Meet Graph Neural Networks: A Perspective of Graph Mining

TL;DR

This paper addresses generalization gaps in graph mining by examining how large language models can enrich graph-structured data. It introduces a three-way taxonomy—GNN-driving-LLM, LLM-driving-GNN, and GNN-LLM-co-driving—and surveys techniques across prompting, encoding, fine-tuning, and joint architectures. The review synthesizes representative works (e.g., GraphFormers, GPT4Graph, GLEM, GRENADE) and analyzes trade-offs between semantic richness and computational cost, noting modest yet consistent gains over baselines. It highlights promising directions in multimodal graph processing, hallucination mitigation, and tackling complex graph tasks to broaden the impact of graph mining.

Abstract

Graph mining is an important area in data mining and machine learning that involves extracting valuable information from graph-structured data. In recent years, significant progress has been made in this field through the development of graph neural networks (GNNs). However, GNNs are still deficient in generalizing to diverse graph data. Aiming to this issue, Large Language Models (LLMs) could provide new solutions for graph mining tasks with their superior semantic understanding. In this review, we systematically review the combination and application techniques of LLMs and GNNs and present a novel taxonomy for research in this interdisciplinary field, which involves three main categories: GNN-driving-LLM, LLM-driving-GNN, and GNN-LLM-co-driving. Within this framework, we reveal the capabilities of LLMs in enhancing graph feature extraction as well as improving the effectiveness of downstream tasks such as node classification, link prediction, and community detection. Although LLMs have demonstrated their great potential in handling graph-structured data, their high computational requirements and complexity remain challenges. Future research needs to continue to explore how to efficiently fuse LLMs and GNNs to achieve more powerful graph learning and reasoning capabilities and provide new impetus for the development of graph mining techniques.

Figures (6)

• Figure 1: Based on the relationship between LLMs and GNNs, we categorize the application scenarios of LLMs in graph mining into three groups: GNN-driving-LLM, LLM-driving-GNN, and GNN-LLM-co-driving. These models are capable of handling a variety of datasets and achieving success in various downstream tasks.
• Figure 2: The proposed taxonomy on LLM-GNN-combined techniques.
• Figure 3: GNN-driving-LLM: a) LLMs generate additional information for the textual attributes of the nodes. These features extracted by LLMs are fed into the next layer of the language model to generate enhanced node embeddings, which are finally processed by GNNs for downstream tasks; b) the textual embeddings exported by LLMs can be directly used as the initial node embeddings for GNNs.
• Figure 4: LLM-driving-GNN: Models use spreading functions or GNNs to transform graph data into sequence text so that LLMs can directly understand the graph data and accomplish downstream tasks.
• Figure 5: a) Text encoding and graph aggregation are iteratively executed through hierarchical Graph Neural Networks (GNN) and Transformers (TRM); b) Data from these two different modalities are projected into an aligned semantic embedding space, where attention mechanisms are used to learn the correlation rules between molecular substructures and textual keywords.