DynLLM: When Large Language Models Meet Dynamic Graph Recommendation

TL;DR

DynLLM addresses dynamic recommendation on continuous-time graphs by integrating CTDG-based temporal/topological signals with LLM-generated, multi-faceted user profiles. It introduces four modules: temporal neighbor aggregation via TGANs, LLM-based multi-facet profile augmentation, a distilled attention mechanism to suppress noise and fuse signals, and a training-update loop that includes GRU-based next-time updates. The approach yields consistent improvements over static and dynamic baselines on real e-commerce datasets, thanks to explicit temporal modeling, rich textual augmentation, and noise-robust fusion. This work demonstrates a practical pathway to leverage LLMs in evolving graph contexts for more accurate and explainable recommendations with potential broad impact on data-sparse, temporally evolving domains.

Abstract

Last year has witnessed the considerable interest of Large Language Models (LLMs) for their potential applications in recommender systems, which may mitigate the persistent issue of data sparsity. Though large efforts have been made for user-item graph augmentation with better graph-based recommendation performance, they may fail to deal with the dynamic graph recommendation task, which involves both structural and temporal graph dynamics with inherent complexity in processing time-evolving data. To bridge this gap, in this paper, we propose a novel framework, called DynLLM, to deal with the dynamic graph recommendation task with LLMs. Specifically, DynLLM harnesses the power of LLMs to generate multi-faceted user profiles based on the rich textual features of historical purchase records, including crowd segments, personal interests, preferred categories, and favored brands, which in turn supplement and enrich the underlying relationships between users and items. Along this line, to fuse the multi-faceted profiles with temporal graph embedding, we engage LLMs to derive corresponding profile embeddings, and further employ a distilled attention mechanism to refine the LLM-generated profile embeddings for alleviating noisy signals, while also assessing and adjusting the relevance of each distilled facet embedding for seamless integration with temporal graph embedding from continuous time dynamic graphs (CTDGs). Extensive experiments on two real e-commerce datasets have validated the superior improvements of DynLLM over a wide range of state-of-the-art baseline methods.

Figures (5)

• Figure 1: The process of generating profile embeddings from LLMs. According to the rich textual titles $\{title_1, title_2, \dots, title_n\}$ from corresponding purchased items $\{i_1, i_2, \dots, i_n\}$ of a user $u$, the multi-faceted profiles are derived from LLM-generation through designed prompt, and then the profile embeddings are generated through LLM-embedding.
• Figure 2: The overall framework of the proposed DynLLM. There are four principal modules: (a) Temporal Neighbors Information Aggregation that aggregates temporal neighbors information for users and items in CTDGS, (b) LLM-based Multi-faceted Information Augmentation that generates multi-faceted user profiles through LLMs, (c) Multi-faceted Distilled Attention Mechanism that distills and incorporate LLM-generated embeddings to enhance representation, and (d) Model Training and Update that trains the model and update embeddings for future recommendation.
• Figure 3: Ablation study on multi-faceted profiles
• Figure 4: Parameters sensitivity of the number of user neighbors and item neighbors
• Figure 5: Parameters sensitivity of the distillation coefficient

Theorems & Definitions (3)

• Definition 1
• Definition 2
• Definition 3