Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Learning and Its Advancements on Large Language Models: A Holistic Survey

Shaopeng Wei, Jun Wang, Yu Zhao, Xingyan Chen, Qing Li, Fuzhen Zhuang, Ji Liu, Fuji Ren, Gang Kou

TL;DR

This survey focuses on the most recent advancements in integrating graph learning with pre-trained language models, specifically emphasizing their application within the domain of large language models.

Abstract

Graph learning is a prevalent domain that endeavors to learn the intricate relationships among nodes and the topological structure of graphs. Over the years, graph learning has transcended from graph theory to graph data mining. With the advent of representation learning, it has attained remarkable performance in diverse scenarios. Owing to its extensive application prospects, graph learning attracts copious attention. While some researchers have accomplished impressive surveys on graph learning, they failed to connect related objectives, methods, and applications in a more coherent way. As a result, they did not encompass current ample scenarios and challenging problems due to the rapid expansion of graph learning. Particularly, large language models have recently had a disruptive effect on human life, but they also show relative weakness in structured scenarios. The question of how to make these models more powerful with graph learning remains open. Our survey focuses on the most recent advancements in integrating graph learning with pre-trained language models, specifically emphasizing their application within the domain of large language models. Different from previous surveys on graph learning, we provide a holistic review that analyzes current works from the perspective of graph structure, and discusses the latest applications, trends, and challenges in graph learning. Specifically, we commence by proposing a taxonomy and then summarize the methods employed in graph learning. We then provide a detailed elucidation of mainstream applications. Finally, we propose future directions.

Graph Learning and Its Advancements on Large Language Models: A Holistic Survey

TL;DR

This survey focuses on the most recent advancements in integrating graph learning with pre-trained language models, specifically emphasizing their application within the domain of large language models.

Abstract

Graph learning is a prevalent domain that endeavors to learn the intricate relationships among nodes and the topological structure of graphs. Over the years, graph learning has transcended from graph theory to graph data mining. With the advent of representation learning, it has attained remarkable performance in diverse scenarios. Owing to its extensive application prospects, graph learning attracts copious attention. While some researchers have accomplished impressive surveys on graph learning, they failed to connect related objectives, methods, and applications in a more coherent way. As a result, they did not encompass current ample scenarios and challenging problems due to the rapid expansion of graph learning. Particularly, large language models have recently had a disruptive effect on human life, but they also show relative weakness in structured scenarios. The question of how to make these models more powerful with graph learning remains open. Our survey focuses on the most recent advancements in integrating graph learning with pre-trained language models, specifically emphasizing their application within the domain of large language models. Different from previous surveys on graph learning, we provide a holistic review that analyzes current works from the perspective of graph structure, and discusses the latest applications, trends, and challenges in graph learning. Specifically, we commence by proposing a taxonomy and then summarize the methods employed in graph learning. We then provide a detailed elucidation of mainstream applications. Finally, we propose future directions.
Paper Structure (60 sections, 15 equations, 8 figures, 4 tables)

This paper contains 60 sections, 15 equations, 8 figures, 4 tables.

Table of Contents

  1. Discussion of the Novelty
  2. Contribution
  3. Objectives
  4. Data
  5. Node Perspective
  6. Edge Perspective
  7. Graph Perspective
  8. Models
  9. Non-Embedding Methods
  10. Embedding-based Methods
  11. Tasks
  12. Node-based Tasks
  13. Edge-based Tasks
  14. Graph-based Tasks
  15. Methods
  16. ...and 45 more sections

Figures (8)

  • Figure 1: Number of GL publications in recent years on top venues. Figures (a) and (b) depict the number of GL publications at top conferences and the total number, respectively.
  • Figure 2: Chronological overview of graph learning methods. Methods in green, blue, red and brown are graph mining methods, graph embedding methods, graph neural network methods and LLM-GLs, respectively.
  • Figure 3: A taxonomy of graph learning by Objectives, Methods, Applications and Future Directions.
  • Figure 4: The comparison between graph and hypergraph Feng2019Hypergraph
  • Figure 5: Current research on the combination of LLMs and GL.
  • ...and 3 more figures

Theorems & Definitions (9)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Definition 7
  • Definition 8
  • Definition 9