Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UniRel-R1: RL-tuned LLM Reasoning for Knowledge Graph Relational Question Answering

Yinxu Tang, Chengsong Huang, Jiaxin Huang, William Yeoh

TL;DR

This work defines relation-centric KGQA, where answers are subgraphs capturing semantic connections among seed entities rather than single entities.It introduces UniRel-R1, a unified pipeline combining subgraph selection, multi-stage pruning using hub penalties, and an RL-tuned LLM with a composite reward that prefers compact, informative subgraphs.Across seven benchmark KG datasets and multiple LLM families (Qwen and Llama), UniRel-R1 achieves large gains in connectivity and reward and demonstrates generalization to unseen entities and relations.The study also analyzes model differences, finding Qwen models rely more on semantic cues while Llama models leverage structural connectivity, and shows scalability to three- and four-entity queries.

Abstract

Knowledge Graph Question Answering (KGQA) has traditionally focused on entity-centric queries that return a single answer entity. However, real-world queries are often relational, seeking to understand how entities are associated. In this work, we introduce relation-centric KGQA, a complementary setting where the answer is a subgraph capturing the semantic connections among entities rather than an individual entity. The main challenge lies in the abundance of candidate subgraphs, where trivial or overly common connections often obscure the identification of unique and informative answers. To tackle this, we propose UniRel-R1, a unified framework that integrates subgraph selection, multi-stage graph pruning, and an LLM fine-tuned with reinforcement learning. The reward function is designed to encourage compact and specific subgraphs with more informative relations and lower-degree intermediate entities. Extensive experiments show that UniRel-R1 achieves significant gains in connectivity and reward over Vanilla baselines and generalizes effectively to unseen entities and relations.

UniRel-R1: RL-tuned LLM Reasoning for Knowledge Graph Relational Question Answering

TL;DR

This work defines relation-centric KGQA, where answers are subgraphs capturing semantic connections among seed entities rather than single entities.It introduces UniRel-R1, a unified pipeline combining subgraph selection, multi-stage pruning using hub penalties, and an RL-tuned LLM with a composite reward that prefers compact, informative subgraphs.Across seven benchmark KG datasets and multiple LLM families (Qwen and Llama), UniRel-R1 achieves large gains in connectivity and reward and demonstrates generalization to unseen entities and relations.The study also analyzes model differences, finding Qwen models rely more on semantic cues while Llama models leverage structural connectivity, and shows scalability to three- and four-entity queries.

Abstract

Knowledge Graph Question Answering (KGQA) has traditionally focused on entity-centric queries that return a single answer entity. However, real-world queries are often relational, seeking to understand how entities are associated. In this work, we introduce relation-centric KGQA, a complementary setting where the answer is a subgraph capturing the semantic connections among entities rather than an individual entity. The main challenge lies in the abundance of candidate subgraphs, where trivial or overly common connections often obscure the identification of unique and informative answers. To tackle this, we propose UniRel-R1, a unified framework that integrates subgraph selection, multi-stage graph pruning, and an LLM fine-tuned with reinforcement learning. The reward function is designed to encourage compact and specific subgraphs with more informative relations and lower-degree intermediate entities. Extensive experiments show that UniRel-R1 achieves significant gains in connectivity and reward over Vanilla baselines and generalizes effectively to unseen entities and relations.

Paper Structure

This paper contains 26 sections, 20 equations, 4 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Framework: UniRel-R1
  4. Subgraph Selection
  5. Multi-step Graph Pruning
  6. Relational Answer Generation via LLM
  7. Reinforcement Learning Algorithms
  8. Reward design
  9. Experiment
  10. Experiment Setup
  11. Main Results
  12. Generalization
  13. Scalability
  14. Related Work
  15. Conclusion
  16. ...and 11 more sections

Figures (4)

  • Figure 1: An overview of KGQA and our proposed UniRel-R1 framework. (a) An illustrative example contrasting entity-centric KGQA with relation-centric KGQA. (b) The UniRel-R1 pipeline, which integrates subgraph selection, graph pruning, and RL-tuned LLM.
  • Figure 2: An example of multi-step graph pruning. From the initial subgraph (left), Stage 1 removes overly common entities (e.g., Female), Stage 2 enforces connectivity through intersections, Stage 3 prunes leaf entities, and Stage 4 enhances compactness by removing high-penalty intersection nodes (yellow). The final subgraph (right) retains the most informative relations among the seed entities.
  • Figure 3: Case studies in Freebase13 and FB15k-237.
  • Figure 4: Comparison of connectivity ratio and average reward under different modification levels.