Temporal Graph Neural Network-Powered Paper Recommendation on Dynamic Citation Networks

TL;DR

Addresses the challenge of recommending relevant papers amid rapid publication growth by injecting a temporal dimension into citation networks and updating embeddings as new citations occur. The authors build a Temporal Graph Neural Network-based recommender (TGN-TRec) with a memory module (GRU-based) and a Graph Transformer for time-aware propagation, using edge timestamps and SciBERT-initialized text features. Evaluations on a PapersWithCode-derived dataset of 313,278 papers and 2,233,780 citations show improvements over static GNN baselines in MRR and Precision/Recall at top ranks. Key contributions include introducing continuous-time dynamic citation modeling, a practical construction workflow for temporal citation graphs, and a TGN-TRec engine for more accurate scholarly paper recommendations.

Abstract

Due to the rapid growth of scientific publications, identifying all related reference articles in the literature has become increasingly challenging yet highly demanding. Existing methods primarily assess candidate publications from a static perspective, focusing on the content of articles and their structural information, such as citation relationships. There is a lack of research regarding how to account for the evolving impact among papers on their embeddings. Toward this goal, this paper introduces a temporal dimension to paper recommendation strategies. The core idea is to continuously update a paper's embedding when new citation relationships appear, enhancing its relevance for future recommendations. Whenever a citation relationship is added to the literature upon the publication of a paper, the embeddings of the two related papers are updated through a Temporal Graph Neural Network (TGN). A learnable memory update module based on a Recurrent Neural Network (RNN) is utilized to study the evolution of the embedding of a paper in order to predict its reference impact in a future timestamp. Such a TGN-based model learns a pattern of how people's views of the paper may evolve, aiming to guide paper recommendations more precisely. Extensive experiments on an open citation network dataset, including 313,278 articles from https://paperswithcode.com/about PaperWithCode, have demonstrated the effectiveness of the proposed approach.

Figures (4)

• Figure 1: Illustration of dynamic citation networks, the graph will incrementally expand as time.
• Figure 2: The illustration of the model processing batch of interactions between nodes $P1$, $P2$ and $P3$, the temporal graph module computes messages for each interaction and using an aggregate function to merge messages that send to each node, the GRU cell will take aggregated messages and previous state/meomory of each node and output is nodes' state/memory in the latest timestamp. For performing transformer convolution operation in the Graph Transformer Module, we use source node state, destination node state and edge attribute(encoded time different) as Q, K, V to a scaled dot-product attention layer. The output node embeddings of the graph transformer module are used for computing citation scores in the Recommendation Module.
• Figure 3: Training Loss evolution over epochs for the TGN-TRec models with initialized and zero-initialized memory.
• Figure 4: Validation MRR, APS, and AUCS evolution over epochs for the TGN-TRec models with initialized and zero-initialized memory.