Towards Adaptive Neighborhood for Advancing Temporal Interaction Graph Modeling

TL;DR

This work tackles the limitation of fixed neighborhood encoding in Temporal Graph Networks by proposing SEAN, a plug-and-play module that enables adaptive, personalized, and temporally aware neighborhood encoding for Temporal Interaction Graphs. SEAN comprises two core components: Representative Neighbor Selector, which uses occurrence-aware attention and a neighbor diversity penalty to identify salient neighbors, and the Temporal-aware Aggregator, which employs an LSTM-based mechanism with adaptive pruning and outdated-decay to fuse neighborhood information over time. Training uses self-supervised temporal link prediction with an auxiliary neighbor-diversity loss, and the method demonstrates consistent gains across multiple backbones (TGN, TIGE, DyGFormer) on five TIG benchmarks, including the new TemFin dataset, achieving state-of-the-art performance and robustness to noise and neighborhood expansion. The results indicate significant improvements in both temporal link prediction and evolving node classification, highlighting SEAN’s potential to enhance real-world temporal graphs by learning flexible, context-aware neighborhood structures with strong interpretability through the pruning mechanism.

Abstract

Temporal Graph Networks (TGNs) have demonstrated their remarkable performance in modeling temporal interaction graphs. These works can generate temporal node representations by encoding the surrounding neighborhoods for the target node. However, an inherent limitation of existing TGNs is their reliance on fixed, hand-crafted rules for neighborhood encoding, overlooking the necessity for an adaptive and learnable neighborhood that can accommodate both personalization and temporal evolution across different timestamps. In this paper, we aim to enhance existing TGNs by introducing an adaptive neighborhood encoding mechanism. We present SEAN, a flexible plug-and-play model that can be seamlessly integrated with existing TGNs, effectively boosting their performance. To achieve this, we decompose the adaptive neighborhood encoding process into two phases: (i) representative neighbor selection, and (ii) temporal-aware neighborhood information aggregation. Specifically, we propose the Representative Neighbor Selector component, which automatically pinpoints the most important neighbors for the target node. It offers a tailored understanding of each node's unique surrounding context, facilitating personalization. Subsequently, we propose a Temporal-aware Aggregator, which synthesizes neighborhood aggregation by selectively determining the utilization of aggregation routes and decaying the outdated information, allowing our model to adaptively leverage both the contextually significant and current information during aggregation. We conduct extensive experiments by integrating SEAN into three representative TGNs, evaluating their performance on four public datasets and one financial benchmark dataset introduced in this paper. The results demonstrate that SEAN consistently leads to performance improvements across all models, achieving SOTA performance and exceptional robustness.

Figures (13)

• Figure 1: Comparison between fixed and adaptive neighborhoods during the encoding process. (a) Existing TGNs tgntigerdygformer always adopt fixed rules for neighborhood encoding, e.g., encode 2-hop neighborhoods. (b) We propose adaptive neighborhood encoding to facilitate both the personalized and temporal understanding of a target node.
• Figure 2: Framework of the proposed plug-and-play model. Our Representative Neighbor Selector can empower the model to pinpoint the important neighbors, who then act as the personalized representatives for the target node. Meanwhile, we propose the neighbor diversity penalty to penalize the over-concentration of these neighbors, thus maintaining a more balanced neighborhood. Furthermore, we conduct our Temporal-aware Aggregator by LSTM aggregation, where we propose an adaptive pruning module that explicitly determines whether to aggregate information from the given route or to prune it as needed, and an outdated-decay mechanism that de-emphasizes the outdated information. Finally, we generate the temporal node representations for downstream tasks by extracting the encoded neighborhood information from the $K$-th layer of SEAN.
• Figure 3: Robustness to noisy neighborhoods on Wikipedia and Reddit in different perturbation rates.
• Figure 4: Attention scores assigned from randomly selected nodes to their perturbed neighbors on noisy Wikipedia.
• Figure 5: Robustness to expanded neighborhoods on MOOC.

Theorems & Definitions (2)

• definition 1
• definition 2