Prompt Learning on Temporal Interaction Graphs

TL;DR

This work tackles gaps in pre-training temporal interaction graphs (TIGs) by introducing TIGPrompt, a lightweight prompting framework that injects temporally-aware prompts per node through a Temporal Prompt Generator (TProG). It presents three TProG variants—Vanilla, Transformer, and Projection—and shows how prompting can bridge temporal drift and semantic differences between pretext and downstream tasks, while optionally enabling prompt-based fine-tuning for greater adaptability. By freezing the backbone TIG models during prompt tuning, the approach achieves state-of-the-art results on link prediction and node classification across four datasets with improved efficiency, and demonstrates strong performance even with limited data. The methodology offers a practical path to deploy TIG models in dynamic environments with varying compute budgets and data availability, highlighting the value of task-specific temporal prompts for real-world systems, with representations formed as $\widetilde{\mathbf{Z}} = f_{\rho}(\mathbf{Z}, \mathbf{P})$.

Abstract

Temporal Interaction Graphs (TIGs) are widely utilized to represent real-world systems. To facilitate representation learning on TIGs, researchers have proposed a series of TIG models. However, these models are still facing two tough gaps between the pre-training and downstream predictions in their ``pre-train, predict'' training paradigm. First, the temporal discrepancy between the pre-training and inference data severely undermines the models' applicability in distant future predictions on the dynamically evolving data. Second, the semantic divergence between pretext and downstream tasks hinders their practical applications, as they struggle to align with their learning and prediction capabilities across application scenarios. Recently, the ``pre-train, prompt'' paradigm has emerged as a lightweight mechanism for model generalization. Applying this paradigm is a potential solution to solve the aforementioned challenges. However, the adaptation of this paradigm to TIGs is not straightforward. The application of prompting in static graph contexts falls short in temporal settings due to a lack of consideration for time-sensitive dynamics and a deficiency in expressive power. To address this issue, we introduce Temporal Interaction Graph Prompting (TIGPrompt), a versatile framework that seamlessly integrates with TIG models, bridging both the temporal and semantic gaps. In detail, we propose a temporal prompt generator to offer temporally-aware prompts for different tasks. These prompts stand out for their minimalistic design, relying solely on the tuning of the prompt generator with very little supervision data. To cater to varying computational resource demands, we propose an extended ``pre-train, prompt-based fine-tune'' paradigm, offering greater flexibility. Through extensive experiments, the TIGPrompt demonstrates the SOTA performance and remarkable efficiency advantages.

Figures (8)

• Figure 1: (a) The "pre-train, predict" paradigm adopted by existing TIG models, which exhibits both temporal and semantic gaps when applied on the downstream task. (b) Our introduced prompting mechanism, with an innovative Temporal Prompt Generator, designed to mitigate both gaps.
• Figure 2: Overview of TIGPrompt: (a) During the prompt tuning stage, the node embedding, calculated by the pre-trained TIG model, is combined with the personalized prompt embedding for downstream tasks. The TProG is optimized during this stage. (b) The key distinction between the two modes lies in whether the parameters of the TIG model are tuned.
• Figure 3: Comparison between traditional prompt on static graphs 03liu2023graphprompt06fang2022universal and our methods ("pre-train, prompt" paradigm, transductive link prediction on Reddit and MOOC datasets).
• Figure 4: Performance w.r.t the Proportion of Prompting Data. This figure is continued in Appendix \ref{['sec:app_para_analysis']}.
• Figure 5: Performance w.r.t the Prompts Dimension. This figure shares the same legend with Fig. \ref{['fig:para_data']} and is continued in Appendix \ref{['sec:app_para_analysis']}.