Retrieval Augmented Generation for Dynamic Graph Modeling

TL;DR

RAG4DyG addresses the challenge of dynamic graph modeling where solely historical interactions limit adaptability to evolving patterns. It proposes a Retrieval-Augmented Generation framework that combines a time- and context-aware retriever with a graph fusion generator to incorporate broader temporal and structural signals, using $\mu(t_q,t_p)=\exp(-\lambda|t_q-t_p|)$ and a loss $\mathcal{L}_{\text{ret}}=\mathcal{L}_{\text{tcl}}+\alpha\mathcal{L}_{\text{ccl}}$ to guide retrieval. A summary graph $G_{\text{fus}}$ is constructed from top-K demonstrations and processed via a GCN to produce a fused representation $e_{\text{fus}}$, which is prepended to the query sequence for generation. Across six real-world datasets, RAG4DyG achieves state-of-the-art performance in both transductive and inductive settings, demonstrating enhanced generalization to unseen nodes and evolving graph structures with practical training efficiency.

Abstract

Modeling dynamic graphs, such as those found in social networks, recommendation systems, and e-commerce platforms, is crucial for capturing evolving relationships and delivering relevant insights over time. Traditional approaches primarily rely on graph neural networks with temporal components or sequence generation models, which often focus narrowly on the historical context of target nodes. This limitation restricts the ability to adapt to new and emerging patterns in dynamic graphs. To address this challenge, we propose a novel framework, Retrieval-Augmented Generation for Dynamic Graph modeling (RAG4DyG), which enhances dynamic graph predictions by incorporating contextually and temporally relevant examples from broader graph structures. Our approach includes a time- and context-aware contrastive learning module to identify high-quality demonstrations and a graph fusion strategy to effectively integrate these examples with historical contexts. The proposed framework is designed to be effective in both transductive and inductive scenarios, ensuring adaptability to previously unseen nodes and evolving graph structures. Extensive experiments across multiple real-world datasets demonstrate the effectiveness of RAG4DyG in improving predictive accuracy and adaptability for dynamic graph modeling. The code and datasets are publicly available at https://github.com/YuxiaWu/RAG4DyG.

Figures (6)

• Figure 1: Illustration of RAG in NLP and dynamic graph modeling. (a) In NLP, RAG leverages pre-trained language models to encode text and retrieve semantically similar or related demonstrations, which are further concatenated to enhance the generation task. (b) Our work addresses the challenges of complex temporal and structural characteristics of dynamic graphs, incorporating RAG through time- and context-aware retrieval and graph fusion modules.
• Figure 2: Overall framework of RAG4DyG. (a) Sequence modeling for dynamic graphs. (b) The retriever finds top-K temporally and contextually relevant demonstrations. (c) Graph fusion integrates the retrieved demonstrations for the subsequent generation.
• Figure 3: Ablation study for retrieval results.
• Figure 4: Ablation study for link prediction results.
• Figure 5: Effect of the number of demonstrations K.