Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks

Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, Pan Li

TL;DR

This work introduces Causal Anonymous Walks (CAWs) to inductively model temporal networks by capturing causality-driven motifs without relying on node identities. The CAW-N encoder processes anonymized CAWs with an online, constant-memory sampling strategy, enabling scalable link prediction that generalizes to unseen nodes. Empirical results across six real temporal networks show state-of-the-art inductive performance and robust transductive results, with ablations validating the importance of the anonymization and encoding components. The approach reduces feature engineering, preserves fine-grained temporal information, and opens avenues for interpretable motif-based analysis of temporal dynamics.

Abstract

Temporal networks serve as abstractions of many real-world dynamic systems. These networks typically evolve according to certain laws, such as the law of triadic closure, which is universal in social networks. Inductive representation learning of temporal networks should be able to capture such laws and further be applied to systems that follow the same laws but have not been unseen during the training stage. Previous works in this area depend on either network node identities or rich edge attributes and typically fail to extract these laws. Here, we propose Causal Anonymous Walks (CAWs) to inductively represent a temporal network. CAWs are extracted by temporal random walks and work as automatic retrieval of temporal network motifs to represent network dynamics while avoiding the time-consuming selection and counting of those motifs. CAWs adopt a novel anonymization strategy that replaces node identities with the hitting counts of the nodes based on a set of sampled walks to keep the method inductive, and simultaneously establish the correlation between motifs. We further propose a neural-network model CAW-N to encode CAWs, and pair it with a CAW sampling strategy with constant memory and time cost to support online training and inference. CAW-N is evaluated to predict links over 6 real temporal networks and uniformly outperforms previous SOTA methods by averaged 10% AUC gain in the inductive setting. CAW-N also outperforms previous methods in 4 out of the 6 networks in the transductive setting.

Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks

TL;DR

This work introduces Causal Anonymous Walks (CAWs) to inductively model temporal networks by capturing causality-driven motifs without relying on node identities. The CAW-N encoder processes anonymized CAWs with an online, constant-memory sampling strategy, enabling scalable link prediction that generalizes to unseen nodes. Empirical results across six real temporal networks show state-of-the-art inductive performance and robust transductive results, with ablations validating the importance of the anonymization and encoding components. The approach reduces feature engineering, preserves fine-grained temporal information, and opens avenues for interpretable motif-based analysis of temporal dynamics.

Abstract

Temporal networks serve as abstractions of many real-world dynamic systems. These networks typically evolve according to certain laws, such as the law of triadic closure, which is universal in social networks. Inductive representation learning of temporal networks should be able to capture such laws and further be applied to systems that follow the same laws but have not been unseen during the training stage. Previous works in this area depend on either network node identities or rich edge attributes and typically fail to extract these laws. Here, we propose Causal Anonymous Walks (CAWs) to inductively represent a temporal network. CAWs are extracted by temporal random walks and work as automatic retrieval of temporal network motifs to represent network dynamics while avoiding the time-consuming selection and counting of those motifs. CAWs adopt a novel anonymization strategy that replaces node identities with the hitting counts of the nodes based on a set of sampled walks to keep the method inductive, and simultaneously establish the correlation between motifs. We further propose a neural-network model CAW-N to encode CAWs, and pair it with a CAW sampling strategy with constant memory and time cost to support online training and inference. CAW-N is evaluated to predict links over 6 real temporal networks and uniformly outperforms previous SOTA methods by averaged 10% AUC gain in the inductive setting. CAW-N also outperforms previous methods in 4 out of the 6 networks in the transductive setting.

Paper Structure

This paper contains 30 sections, 3 theorems, 16 equations, 10 figures, 8 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation and Notations
  4. Proposed Method: Causal Anonymous Walk-Network
  5. Preliminaries: Anonymous Walk and Temporal Network Motif
  6. Causal Anonymous Walk
  7. Neural Encoding for Causal Anonymous Walks
  8. Extension and Discussion
  9. experiments
  10. Experimental Setup
  11. Results and Discussion
  12. Hyperparameter Investigation of CAW Sampling
  13. Complexity Evaluation
  14. Conclusion
  15. Efficient link Sampling
  16. ...and 15 more sections

Key Result

Theorem 4.1

For two pairs of walk sets $\{S_u, S_v\}$ and $\{S_{u'}, S_{v'}\}$, if there exists a bijective mapping $\pi$ between node identities such that each walk $W$ in $S_u\cup S_v$ can be bijectively mapped to one walk $W'$ in $S_{u'}\cup S_{v'}$ according to $\pi(W[i][0]) = W'[i][0]$ for all $i\in [0,m]$

Figures (10)

  • Figure 1: Triadic closure and feed-forward loops: Causal anonymous walks (CAW) capture the laws.
  • Figure 2: Causal anonymous walks (CAW): causality extraction and set-based anonymization.
  • Figure 3: Ambiguity due to removing node identities in TGAT xu2020inductive ($t_1<t_2<t_3$).
  • Figure 4: The correlation between walks needs to be captured to learn this law.
  • Figure 5: Hyperparameter sensitivity in CAW sampling. AUC on all inductive test links are reported.
  • ...and 5 more figures

Theorems & Definitions (5)

  • Theorem 4.1
  • Proposition A.1
  • proof
  • Theorem A.2
  • proof