Query-Specific GNN: A Comprehensive Graph Representation Learning Method for Retrieval Augmented Generation

TL;DR

This work tackles the difficulty of multi-hop retrieval in retrieval-augmented generation by introducing a Multi-information Level Knowledge Graph (Multi-L KG) to capture cross-level semantic relations. A novel Query-Specific Graph Neural Network (QSGNN) then performs intra-level and inter-level message passing guided by the query to produce robust, noise-resistant representations for retrieval. The approach is complemented by synthesized QA data for pre-training and subsequent fine-tuning on human-annotated data, achieving strong gains on high-hop questions (up to 33.8% in certain metrics). Empirically, QSGNN outperforms state-of-the-art KG-RAG baselines across three multi-hop benchmarks, demonstrating the practical value of multi-level, query-aligned graph representations for grounding LLM-generated answers in external knowledge.

Abstract

Retrieval-augmented generation (RAG) has demonstrated its ability to enhance Large Language Models (LLMs) by integrating external knowledge sources. However, multi-hop questions, which require the identification of multiple knowledge targets to form a synthesized answer, raise new challenges for RAG systems. Under the multi-hop settings, existing methods often struggle to fully understand the questions with complex semantic structures and are susceptible to irrelevant noise during the retrieval of multiple information targets. To address these limitations, we propose a novel graph representation learning framework for multi-hop question retrieval. We first introduce a Multi-information Level Knowledge Graph (Multi-L KG) to model various information levels for a more comprehensive understanding of multi-hop questions. Based on this, we design a Query-Specific Graph Neural Network (QSGNN) for representation learning on the Multi-L KG. QSGNN employs intra/inter-level message passing mechanisms, and in each message passing the information aggregation is guided by the query, which not only facilitates multi-granular information aggregation but also significantly reduces the impact of noise. To enhance its ability to learn robust representations, we further propose two synthesized data generation strategies for pre-training the QSGNN. Extensive experimental results demonstrate the effectiveness of our framework in multi-hop scenarios, especially in high-hop questions the improvement can reach 33.8\%. The code is available at: https://github.com/Jerry2398/QSGNN.

Figures (14)

• Figure 1: Performance on multi-hop questions. Existing methods struggle to perform well as hop number increases while QSGNN can achieve better performance on high-hop questions.
• Figure 2: Framework overview. It first constructs Multi-L KG to model the multi-level relationships within corpora. QSGNN is designed to aggregate information from different levels, all the aggregations are guided by query. After pre-training and fine-tuning, it can generate representations for multi-hop questions.
• Figure 3: Influence of pre-training scale on various model dimension.
• Figure 4: Good case 1 for QSGNN.
• Figure 5: Good case 2 for QSGNN.