Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generating Causal Temporal Interaction Graphs for Counterfactual Validation of Temporal Link Prediction

Aniq Ur Rahman, Justin P. Coon

TL;DR

The paper tackles the problem that temporal link prediction models are usually evaluated only on predictive accuracy, not on whether they capture the causal dynamics of temporal interactions. It introduces a structural equation model for continuous-time event sequences with excitatory and inhibitory effects and extends this to causal temporal interaction graphs CTIGs with known ground truth. A cross-model predictive-error distance $\bar{d}_{A,B}$ is defined to quantify causal distance between generating models, enabling counterfactual evaluation under controlled causal shifts and timestamp distortions. Empirical results show that the TGN model is causally sensitive under these tests while JODIE is not, highlighting the value of causality-aware benchmarking for TLP and providing a framework that can be extended to other domains.

Abstract

Temporal link prediction (TLP) models are commonly evaluated based on predictive accuracy, yet such evaluations do not assess whether these models capture the causal mechanisms that govern temporal interactions. In this work, we propose a framework for counterfactual validation of TLP models by generating causal temporal interaction graphs (CTIGs) with known ground-truth causal structure. We first introduce a structural equation model for continuous-time event sequences that supports both excitatory and inhibitory effects, and then extend this mechanism to temporal interaction graphs. To compare causal models, we propose a distance metric based on cross-model predictive error, and empirically validate the hypothesis that predictors trained on one causal model degrade when evaluated on sufficiently distant models. Finally, we instantiate counterfactual evaluation under (i) controlled causal shifts between generating models and (ii) timestamp shuffling as a stochastic distortion with measurable causal distance. Our framework provides a foundation for causality-aware benchmarking.

Generating Causal Temporal Interaction Graphs for Counterfactual Validation of Temporal Link Prediction

TL;DR

The paper tackles the problem that temporal link prediction models are usually evaluated only on predictive accuracy, not on whether they capture the causal dynamics of temporal interactions. It introduces a structural equation model for continuous-time event sequences with excitatory and inhibitory effects and extends this to causal temporal interaction graphs CTIGs with known ground truth. A cross-model predictive-error distance is defined to quantify causal distance between generating models, enabling counterfactual evaluation under controlled causal shifts and timestamp distortions. Empirical results show that the TGN model is causally sensitive under these tests while JODIE is not, highlighting the value of causality-aware benchmarking for TLP and providing a framework that can be extended to other domains.

Abstract

Temporal link prediction (TLP) models are commonly evaluated based on predictive accuracy, yet such evaluations do not assess whether these models capture the causal mechanisms that govern temporal interactions. In this work, we propose a framework for counterfactual validation of TLP models by generating causal temporal interaction graphs (CTIGs) with known ground-truth causal structure. We first introduce a structural equation model for continuous-time event sequences that supports both excitatory and inhibitory effects, and then extend this mechanism to temporal interaction graphs. To compare causal models, we propose a distance metric based on cross-model predictive error, and empirically validate the hypothesis that predictors trained on one causal model degrade when evaluated on sufficiently distant models. Finally, we instantiate counterfactual evaluation under (i) controlled causal shifts between generating models and (ii) timestamp shuffling as a stochastic distortion with measurable causal distance. Our framework provides a foundation for causality-aware benchmarking.
Paper Structure (19 sections, 7 theorems, 28 equations, 5 figures, 1 table, 1 algorithm)

This paper contains 19 sections, 7 theorems, 28 equations, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Organisation
  3. Notation
  4. Causal Event Sequence
  5. Proposed Model
  6. Properties of the proposed causal model
  7. Related Works
  8. Comparing Causal Models
  9. Causal Temporal Interaction Graphs
  10. Counterfactual Analysis
  11. Motivation
  12. Application
  13. Negative Sampling
  14. Experiment A
  15. Experiment B
  16. ...and 4 more sections

Key Result

proposition 1

The causal model $( \mathscr{G}, \Theta_{\mathscr{G}} )$ is semi-Markovian for all ${\mathbf{A}} \in \{0,1\}^{n \times n}$.

Figures (5)

  • Figure 1: Construction of the CTIG causal parameters for a graph with $n=5$ and $r=5$.
  • Figure 2: Empirical evaluation of the performance gap under causal distortion. The panels visualize how the performance gap $\Delta_\star^{0, \dagger}$ and its associated probabilities vary with the causal distance $\bar{d}_{0, \dagger}$ and the accuracy threshold $\delta_*$, providing support for Hypothesis \ref{['hypo:discrepancy']}. $n=7, T=10^3$.
  • Figure 3: Counterfactual evaluation of TLP models. Each panel shows a scatter plot of the performance gap, overlaid with a LOWESS smoothing curve (fraction $0.95$) to highlight the overall trend.
  • Figure 4: Violin plots of model performance under timestamp shuffling.
  • Figure 5: Causal model.

Theorems & Definitions (17)

  • definition 1: Point Process, haenggi2012stochastic
  • definition 2: 1-D Poisson Point Process, haenggi2012stochastic
  • definition 3: Causal model, pearl2009causality
  • definition 4: Monotonicity, pearl2009causality
  • definition 5: Exogeneity, pearl2009causality
  • proposition 1
  • proof
  • proposition 2
  • proof
  • corollary 1
  • ...and 7 more