Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GraphSOS: Graph Sampling and Order Selection to Help LLMs Understand Graphs Better

Xu Chu, Hanlin Xue, Zhijie Tan, Bingce Wang, Tong Mo, Weiping Li

TL;DR

GraphSOS tackles two core problems in Graph LLMs: sensitivity to input serialization and the use of potentially suboptimal random subgraphs under limited context. It introduces a Subgraph Sampling Module to form informative $2$-hop neighborhoods, an Order Selector Module to generate and select effective input orders, and Graph CoT distilled from GPT-4o via two-stage instruction tuning to enhance graph reasoning. Empirical results on text-attributed graph node classification and graph QA show strong supervised and zero-shot performance gains, with ablations confirming the contribution of each component. The work advances graph understanding in LLMs by combining structured input construction with explicit reasoning, offering practical benefits for scalable graph reasoning in real-world tasks.

Abstract

The success of Large Language Models (LLMs) in various domains has led researchers to apply them to graph-related problems by converting graph data into natural language text. However, unlike graph data, natural language inherently has sequential order. We observe a counter-intuitive fact that when the order of nodes or edges in the natural language description of a graph is shuffled, despite describing the same graph, model performance fluctuates between high performance and random guessing. Additionally, due to LLMs' limited input context length, current methods typically randomly sample neighbors of target nodes as representatives of their neighborhood, which may not always be effective for accurate reasoning. To address these gaps, we introduce GraphSOS (Graph Sampling and Order Selection). This novel model framework features an Order Selector Module to ensure proper serialization order of the graph and a Subgraph Sampling Module to sample subgraphs with better structure for better reasoning. Furthermore, we propose Graph CoT obtained through distillation, and enhance LLM's reasoning and zero-shot learning capabilities for graph tasks through instruction tuning. Experiments on multiple datasets for node classification and graph question-answering demonstrate that GraphSOS improves LLMs' performance and generalization ability on graph tasks.

GraphSOS: Graph Sampling and Order Selection to Help LLMs Understand Graphs Better

TL;DR

GraphSOS tackles two core problems in Graph LLMs: sensitivity to input serialization and the use of potentially suboptimal random subgraphs under limited context. It introduces a Subgraph Sampling Module to form informative -hop neighborhoods, an Order Selector Module to generate and select effective input orders, and Graph CoT distilled from GPT-4o via two-stage instruction tuning to enhance graph reasoning. Empirical results on text-attributed graph node classification and graph QA show strong supervised and zero-shot performance gains, with ablations confirming the contribution of each component. The work advances graph understanding in LLMs by combining structured input construction with explicit reasoning, offering practical benefits for scalable graph reasoning in real-world tasks.

Abstract

The success of Large Language Models (LLMs) in various domains has led researchers to apply them to graph-related problems by converting graph data into natural language text. However, unlike graph data, natural language inherently has sequential order. We observe a counter-intuitive fact that when the order of nodes or edges in the natural language description of a graph is shuffled, despite describing the same graph, model performance fluctuates between high performance and random guessing. Additionally, due to LLMs' limited input context length, current methods typically randomly sample neighbors of target nodes as representatives of their neighborhood, which may not always be effective for accurate reasoning. To address these gaps, we introduce GraphSOS (Graph Sampling and Order Selection). This novel model framework features an Order Selector Module to ensure proper serialization order of the graph and a Subgraph Sampling Module to sample subgraphs with better structure for better reasoning. Furthermore, we propose Graph CoT obtained through distillation, and enhance LLM's reasoning and zero-shot learning capabilities for graph tasks through instruction tuning. Experiments on multiple datasets for node classification and graph question-answering demonstrate that GraphSOS improves LLMs' performance and generalization ability on graph tasks.
Paper Structure (19 sections, 9 equations, 11 figures, 7 tables)

This paper contains 19 sections, 9 equations, 11 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Methodology
  4. Subgraph Sampling Module
  5. Order Selector Module
  6. Graph Chain-of-Thought Distillation
  7. Experiments
  8. Experimental Setup
  9. Overall Performance of GraphSOS (RQ1)
  10. Module Ablation Study (RQ2)
  11. Conclusion
  12. Impact Statement
  13. Analyzing Order Sensitivity in LLMs and LLMs for Graph Tasks
  14. Experimental Setup
  15. Additional Results
  16. ...and 4 more sections

Figures (11)

  • Figure 1: Converting a graph into natural language description. Elements in both node and edge lists can be arranged in any order to represent the same graph.
  • Figure 2: Zero-shot performance of models with different orders of node and edge.
  • Figure 3: The overall framework of GraphSOS. Trainable components are highlighted in yellow and marked with flame icons. (a) Subgraph Sampling Module: samples and outputs a subgraph of a target node from graph $\mathcal{G}$. (b) Order Selector Module: takes the subgraph and user question, converts the subgraph into a text sequence and selects the sequence order. (c) LLM: generates answers based on question and serialized text representation of the graph.
  • Figure 4: Internal details of the Subgraph Sampling Module (SSM). Frozen components are highlighted in blue and marked with snowflake icons.
  • Figure 5: Internal details of the Order Selector Module (OSM). Frozen components are highlighted in blue and marked with snowflake icons.
  • ...and 6 more figures