Exploring the Potential of Large Language Models as Predictors in Dynamic Text-Attributed Graphs

TL;DR

This work tackles predictive tasks on dynamic text-attributed graphs (DyTAGs) by introducing GraphAgent-Dynamic (GAD), a multi-agent framework that uses global and local knowledge agents along with a knowledge reflection module to guide a predictor without dataset-specific training. GAD addresses key challenges of DyTAGs—context-length constraints and domain variability—by decomposing knowledge generation and incorporating adaptive updates, enabling LLM-based predictors to match or surpass full-supervised GNNs on several tasks. Empirical results across five real-world datasets show GAD's strong generalization for future link prediction and node retrieval, with limitations in edge classification under unified rules and recall-dependent tasks. The paper also discusses potential improvements, including dataset-specific fine-tuning and better recall mechanisms, to further enhance LLM-based predictors in dynamic graphs.

Abstract

With the rise of large language models (LLMs), there has been growing interest in Graph Foundation Models (GFMs) for graph-based tasks. By leveraging LLMs as predictors, GFMs have demonstrated impressive generalizability across various tasks and datasets. However, existing research on LLMs as predictors has predominantly focused on static graphs, leaving their potential in dynamic graph prediction unexplored. In this work, we pioneer using LLMs for predictive tasks on dynamic graphs. We identify two key challenges: the constraints imposed by context length when processing large-scale historical data and the significant variability in domain characteristics, both of which complicate the development of a unified predictor. To address these challenges, we propose the GraphAgent-Dynamic (GAD) Framework, a multi-agent system that leverages collaborative LLMs. In contrast to using a single LLM as the predictor, GAD incorporates global and local summary agents to generate domain-specific knowledge, enhancing its transferability across domains. Additionally, knowledge reflection agents enable adaptive updates to GAD's knowledge, maintaining a unified and self-consistent architecture. In experiments, GAD demonstrates performance comparable to or even exceeds that of full-supervised graph neural networks without dataset-specific training. Finally, to enhance the task-specific performance of LLM-based predictors, we discuss potential improvements, such as dataset-specific fine-tuning to LLMs. By developing tailored strategies for different tasks, we provide new insights for the future design of LLM-based predictors.

Figures (8)

• Figure 1: Overview of the GAD pipeline. The Predictor Agent utilizes generated knowledge and relevant metrics to make predictions. The database is updated over time, while the knowledge is reused unless updated through reflection.
• Figure 2: Performance comparison between GNNs (average over TCL, GraphMixer, DygFormer) and LLM-based predictors (average over GPT and DeepSeek-based backbones).
• Figure : Enron
• Figure : Enron
• Figure : GDELT