Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Document Retrieval with G-Retriever

Manthankumar Solanki

TL;DR

This work tackles efficient QA over textual graphs by replacing the PCST-based subgraph construction with an attention-driven retrieval mechanism and by encoding both node and edge attributes. It introduces an enhanced graph encoder with joint node-edge representations, followed by multi-head attention pooling and an improved projection layer to better align with Large Language Models. On WebQSP, the proposed approach achieves competitive, marginally better results than the previous G-Retriever, demonstrating improved context-aware retrieval and graph representations. The method advances practical, scalable retrieval for graph-structured QA by delivering stronger grounding and alignment with LLMs.

Abstract

Textual data question answering has gained significant attention due to its growing applicability. Recently, a novel approach leveraging the Retrieval-Augmented Generation (RAG) method was introduced, utilizing the Prize-Collecting Steiner Tree (PCST) optimization for sub-graph construction. However, this method focused solely on node attributes, leading to incomplete contextual understanding. In this paper, we propose an enhanced approach that replaces the PCST method with an attention-based sub-graph construction technique, enabling more efficient and context-aware retrieval. Additionally, we encode both node and edge attributes, leading to richer graph representations. Our method also incorporates an improved projection layer and multi-head attention pooling for better alignment with Large Language Models (LLMs). Experimental evaluations on the WebQSP dataset demonstrate that our approach is competitive and achieves marginally better results compared to the original method, underscoring its potential for more accurate question answering.

Efficient Document Retrieval with G-Retriever

TL;DR

This work tackles efficient QA over textual graphs by replacing the PCST-based subgraph construction with an attention-driven retrieval mechanism and by encoding both node and edge attributes. It introduces an enhanced graph encoder with joint node-edge representations, followed by multi-head attention pooling and an improved projection layer to better align with Large Language Models. On WebQSP, the proposed approach achieves competitive, marginally better results than the previous G-Retriever, demonstrating improved context-aware retrieval and graph representations. The method advances practical, scalable retrieval for graph-structured QA by delivering stronger grounding and alignment with LLMs.

Abstract

Textual data question answering has gained significant attention due to its growing applicability. Recently, a novel approach leveraging the Retrieval-Augmented Generation (RAG) method was introduced, utilizing the Prize-Collecting Steiner Tree (PCST) optimization for sub-graph construction. However, this method focused solely on node attributes, leading to incomplete contextual understanding. In this paper, we propose an enhanced approach that replaces the PCST method with an attention-based sub-graph construction technique, enabling more efficient and context-aware retrieval. Additionally, we encode both node and edge attributes, leading to richer graph representations. Our method also incorporates an improved projection layer and multi-head attention pooling for better alignment with Large Language Models (LLMs). Experimental evaluations on the WebQSP dataset demonstrate that our approach is competitive and achieves marginally better results compared to the original method, underscoring its potential for more accurate question answering.

Paper Structure

This paper contains 17 sections, 9 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Graphs and Large Language Models (LLMs)
  4. Retrieval-Augmented Generation
  5. Attention mechanism
  6. Method
  7. Retrieval via Attention Mechanism
  8. Enhanced Graph Encoder
  9. Joint Node-Edge Encoding:
  10. Multi-head Attention Pooling:
  11. Enhanced Projection Layer:
  12. Experiments
  13. Reproducible Results
  14. Experiments with Model Architecture
  15. Conclusion
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Retrieval via attention
  • Figure 2: Training loss comparison between Our Method (combined), Our Method (Retrieval via Attention), and the Original Method. Our proposed approach consistently shows lower training loss, indicating better optimization and convergence.
  • Figure 3: Validation loss comparison among different methods. Our proposed method exhibits a lower validation loss, confirming improved generalization performance.