What Do LLMs Need to Understand Graphs: A Survey of Parametric Representation of Graphs
Dongqi Fu, Liri Fang, Zihao Li, Hanghang Tong, Vetle I. Torvik, Jingrui He
TL;DR
Enables LLMs to understand graphs without enumerating complex topology by surveying graph laws as parametric representations. The paper catalogs macroscopic and microscopic, static and dynamic, high- and low-order laws, and introduces new observation spaces and parameters to capture graph structure, guiding tasks such as graph generation, link prediction, and LLM-assisted reasoning. Key contributions include a comprehensive taxonomy of classic and novel graph laws, discussion of law-guided downstream tasks, and considerations for transferring and applying these laws across domains and LLMs. This parametric formalism promises to bridge complex graph topology and NL-based reasoning, with practical impact in molecule design, protein research, and recommender systems by enabling knowledge distillation, retrieval augmentation, and direct graph-based problem solving.
Abstract
Graphs, as a relational data structure, have been widely used for various application scenarios, like molecule design and recommender systems. Recently, large language models (LLMs) are reorganizing in the AI community for their expected reasoning and inference abilities. Making LLMs understand graph-based relational data has great potential, including but not limited to (1) distillate external knowledge base for eliminating hallucination and breaking the context window limit for LLMs' inference during the retrieval augmentation generation process; (2) taking graph data as the input and directly solve the graph-based research tasks like protein design and drug discovery. However, inputting the entire graph data to LLMs is not practical due to its complex topological structure, data size, and the lack of effective and efficient semantic graph representations. A natural question arises: Is there a kind of graph representation that can be described by natural language for LLM's understanding and is also easy to require to serve as the raw input for LLMs? Based on statistical computation, graph laws pre-define a set of parameters (e.g., degree, time, diameter) and identifie their relationships and values by observing the topological distribution of plenty of real-world graph data. We believe this kind of parametric representation of graphs, graph laws, can be a solution for making LLMs understand graph data as the input. In this survey, we first review the previous study of graph laws from multiple perspectives, i.e., macroscope and microscope of graphs, low-order and high-order graphs, static and dynamic graphs, different observation spaces, and newly proposed graph parameters. After we review various real-world applications benefiting from the guidance of graph laws, we conclude the paper with current challenges and future research directions.