Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph-S3: Enhancing Agentic textual Graph Retrieval with Synthetic Stepwise Supervision

Ge Chang, Jinbo Su, Jiacheng Liu, Pengfei Yang, Yuhao Shang, Huiwen Zheng, Hongli Ma, Yan Liang, Yuanchun Li, Yunxin Liu

TL;DR

This paper addresses the challenge of retrieving informative yet compact content from textual graphs for LLM-based QA. It introduces Graph-S3, an agentic retriever trained with synthetic stepwise supervision via a data-synthesis pipeline and a two-stage training regime (SFT followed by RL with stepwise rewards), plus an interactive retrieval mechanism. Empirical results on WebQSP, CWQ, and MetaQA show state-of-the-art performance and substantially reduced retrieval size, especially on multi-hop tasks. The approach demonstrates that fine-grained, stepwise feedback and structured exploration can significantly improve reasoning over textual graphs and the efficiency of retrieval-augmented QA systems.

Abstract

A significant portion of real-world data is inherently represented as textual graphs, and integrating these graphs into large language models (LLMs) is promising to enable complex graph-based question answering. However, a key challenge in LLM-based textual graph QA systems lies in graph retrieval, i.e., how to retrieve relevant content from large graphs that is sufficiently informative while remaining compact for the LLM context. Existing retrievers suffer from poor performance since they either rely on shallow embedding similarity or employ interactive retrieving policies that demand excessive data labeling and training cost. To address these issues, we present Graph-$S^3$, an agentic textual graph reasoning framework that employs an LLM-based retriever trained with synthetic stepwise supervision. Instead of rewarding the agent based on the final answers, which may lead to sparse and unstable training signals, we propose to closely evaluate each step of the retriever based on offline-extracted golden subgraphs. Our main techniques include a data synthesis pipeline to extract the golden subgraphs for reward generation and a two-stage training scheme to learn the interactive graph exploration policy based on the synthesized rewards. Based on extensive experiments on three common datasets in comparison with seven strong baselines, our approach achieves an average improvement of 8.1\% in accuracy and 9.7\% in F$_1$ score. The advantage is even higher in more complicated multi-hop reasoning tasks. Our code will be open-sourced.

Graph-S3: Enhancing Agentic textual Graph Retrieval with Synthetic Stepwise Supervision

TL;DR

This paper addresses the challenge of retrieving informative yet compact content from textual graphs for LLM-based QA. It introduces Graph-S3, an agentic retriever trained with synthetic stepwise supervision via a data-synthesis pipeline and a two-stage training regime (SFT followed by RL with stepwise rewards), plus an interactive retrieval mechanism. Empirical results on WebQSP, CWQ, and MetaQA show state-of-the-art performance and substantially reduced retrieval size, especially on multi-hop tasks. The approach demonstrates that fine-grained, stepwise feedback and structured exploration can significantly improve reasoning over textual graphs and the efficiency of retrieval-augmented QA systems.

Abstract

A significant portion of real-world data is inherently represented as textual graphs, and integrating these graphs into large language models (LLMs) is promising to enable complex graph-based question answering. However, a key challenge in LLM-based textual graph QA systems lies in graph retrieval, i.e., how to retrieve relevant content from large graphs that is sufficiently informative while remaining compact for the LLM context. Existing retrievers suffer from poor performance since they either rely on shallow embedding similarity or employ interactive retrieving policies that demand excessive data labeling and training cost. To address these issues, we present Graph-, an agentic textual graph reasoning framework that employs an LLM-based retriever trained with synthetic stepwise supervision. Instead of rewarding the agent based on the final answers, which may lead to sparse and unstable training signals, we propose to closely evaluate each step of the retriever based on offline-extracted golden subgraphs. Our main techniques include a data synthesis pipeline to extract the golden subgraphs for reward generation and a two-stage training scheme to learn the interactive graph exploration policy based on the synthesized rewards. Based on extensive experiments on three common datasets in comparison with seven strong baselines, our approach achieves an average improvement of 8.1\% in accuracy and 9.7\% in F score. The advantage is even higher in more complicated multi-hop reasoning tasks. Our code will be open-sourced.

Paper Structure

This paper contains 28 sections, 9 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Graph Retrieval Methods
  4. Stepwise Reinforcement Learning for Graph Reasoning
  5. Method
  6. Data Synthesis
  7. Action Space for Graph Exploration
  8. Graph Reasoning Data Synthesis
  9. Answer and retention.
  10. Trajectory Refinement for Reinforcement Learning
  11. Training Stage
  12. Stage I: Supervised fine-tuning
  13. Stage II: Reinforcement learning with stepwise rewards
  14. Interactive Retriever
  15. Experiments
  16. ...and 13 more sections

Figures (3)

  • Figure 1: An illustration of agentic textual graph retrieval and question answering.
  • Figure 2: An Overview of our data synthesis pipeline and training scheme.
  • Figure 3: Number of retrieved graph triples in Graph-S$^3$ and baselines on correct answers.