DynaGRAG | Exploring the Topology of Information for Advancing Language Understanding and Generation in Graph Retrieval-Augmented Generation

TL;DR

DynaGRAG tackles the challenge of integrating rich textual semantics with graph topology in retrieval-augmented generation by introducing a dynamic, density-aware GRAG framework. It preserves graph structure, enhances subgraph representations through de-duplication and two-step mean pooling, and uses a diversity-aware, query-focused retrieval powered by a Dynamic Similarity-Aware BFS, refined by GCNs and hard prompting to enable real-time traversal with LLMs. Key contributions include novel graph consolidation, diversity-prioritized subgraph retrieval, dynamic traversal, soft masking via GCNs, and hierarchical prompting that jointly leverage textual and topological signals. Empirical results on podcast transcripts show DynaGRAG outperforms Vanilla LLM and Naïve RAG across multiple models, evidencing improved reasoning depth, coherence, and contextual coverage with scalable, interpretable graph-based reasoning. Overall, DynaGRAG offers a practical, scalable path to more nuanced and trustworthy AI by tightly coupling graph structure with large-language reasoning without requiring LLM fine-tuning.

Abstract

Graph Retrieval-Augmented Generation (GRAG or Graph RAG) architectures aim to enhance language understanding and generation by leveraging external knowledge. However, effectively capturing and integrating the rich semantic information present in textual and structured data remains a challenge. To address this, a novel GRAG framework, Dynamic Graph Retrieval-Agumented Generation (DynaGRAG), is proposed to focus on enhancing subgraph representation and diversity within the knowledge graph. By improving graph density, capturing entity and relation information more effectively, and dynamically prioritizing relevant and diverse subgraphs and information within them, the proposed approach enables a more comprehensive understanding of the underlying semantic structure. This is achieved through a combination of de-duplication processes, two-step mean pooling of embeddings, query-aware retrieval considering unique nodes, and a Dynamic Similarity-Aware BFS (DSA-BFS) traversal algorithm. Integrating Graph Convolutional Networks (GCNs) and Large Language Models (LLMs) through hard prompting further enhances the learning of rich node and edge representations while preserving the hierarchical subgraph structure. Experimental results demonstrate the effectiveness of DynaGRAG, showcasing the significance of enhanced subgraph representation and diversity for improved language understanding and generation.

Figures (8)

• Figure 1: Radar plot illustrating performance across evaluation metrics for architectures tested using Gemini 1.5 Flash, highlighting DynaGRAG's strengths with empowerment, subjectivity and nuance, and implication focus.
• Figure 2: Radar plot illustrating performance across evaluation metrics for architectures tested using GPT 4-o mini, highlighting DynaGRAG's strengths across all metrics collectively.
• Figure 3: Word cloud representing an LLM's evaluation of the proposed framework's responses, highlighting attributes such as understanding, depth, clarity, focus on ethical and societal implications, and complex reasoning.
• Figure 4: Process diagram describing the graph retrieval and generation process in DynaGRAG.
• Figure 5: Visualization of a knowledge graph extended to a 3D representation based on nodes and entities extracted from transcripts of Dwarkesh Patel's podcast.DwarkeshPodcast:2025 Nodes are positioned using the Kamada-Kawai layout, dynamically scaled, and perturbed along the z-axis for enhanced visual differentiation.