Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Negative-Aware Diffusion Process for Temporal Knowledge Graph Extrapolation

Yanglei Gan, Peng He, Yuxiang Cai, Run Lin, Guanyu Zhou, Qiao Liu

TL;DR

NADEx addresses two core gaps in diffusion-based Temporal Knowledge Graph extrapolation: neglect of informative negative context and reliance on generic reconstruction losses. It encodes subject-centric histories, applies forward diffusion to target embeddings, and uses a Transformer-based denoiser conditioned on temporal-relational context, augmented by a cosine-alignment regularizer built from batch-wise negative prototypes. Empirical results on ICEWS14, ICEWS18, ICEWS05-15, and GDELT show state-of-the-art performance, with strong gains on unseen events and data-scarce scenarios, while maintaining efficient inference. This work advances generative TK reasoning by producing calibrated, discriminative predictions that effectively model temporal dynamics under uncertainty.

Abstract

Temporal Knowledge Graph (TKG) reasoning seeks to predict future missing facts from historical evidence. While diffusion models (DM) have recently gained attention for their ability to capture complex predictive distributions, two gaps remain: (i) the generative path is conditioned only on positive evidence, overlooking informative negative context, and (ii) training objectives are dominated by cross-entropy ranking, which improves candidate ordering but provides little supervision over the calibration of the denoised embedding. To bridge this gap, we introduce Negative-Aware Diffusion model for TKG Extrapolation (NADEx). Specifically, NADEx encodes subject-centric histories of entities, relations and temporal intervals into sequential embeddings. NADEx perturbs the query object in the forward process and reconstructs it in reverse with a Transformer denoiser conditioned on the temporal-relational context. We further derive a cosine-alignment regularizer derived from batch-wise negative prototypes, which tightens the decision boundary against implausible candidates. Comprehensive experiments on four public TKG benchmarks demonstrate that NADEx delivers state-of-the-art performance.

Negative-Aware Diffusion Process for Temporal Knowledge Graph Extrapolation

TL;DR

NADEx addresses two core gaps in diffusion-based Temporal Knowledge Graph extrapolation: neglect of informative negative context and reliance on generic reconstruction losses. It encodes subject-centric histories, applies forward diffusion to target embeddings, and uses a Transformer-based denoiser conditioned on temporal-relational context, augmented by a cosine-alignment regularizer built from batch-wise negative prototypes. Empirical results on ICEWS14, ICEWS18, ICEWS05-15, and GDELT show state-of-the-art performance, with strong gains on unseen events and data-scarce scenarios, while maintaining efficient inference. This work advances generative TK reasoning by producing calibrated, discriminative predictions that effectively model temporal dynamics under uncertainty.

Abstract

Temporal Knowledge Graph (TKG) reasoning seeks to predict future missing facts from historical evidence. While diffusion models (DM) have recently gained attention for their ability to capture complex predictive distributions, two gaps remain: (i) the generative path is conditioned only on positive evidence, overlooking informative negative context, and (ii) training objectives are dominated by cross-entropy ranking, which improves candidate ordering but provides little supervision over the calibration of the denoised embedding. To bridge this gap, we introduce Negative-Aware Diffusion model for TKG Extrapolation (NADEx). Specifically, NADEx encodes subject-centric histories of entities, relations and temporal intervals into sequential embeddings. NADEx perturbs the query object in the forward process and reconstructs it in reverse with a Transformer denoiser conditioned on the temporal-relational context. We further derive a cosine-alignment regularizer derived from batch-wise negative prototypes, which tightens the decision boundary against implausible candidates. Comprehensive experiments on four public TKG benchmarks demonstrate that NADEx delivers state-of-the-art performance.
Paper Structure (33 sections, 22 equations, 4 figures, 7 tables)

This paper contains 33 sections, 22 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Discriminative TKG Reasoning
  4. Generative TKG Reasoning
  5. Preliminary
  6. Method
  7. Temporal Representation Learning
  8. Negative Sampling Strategy
  9. Forward Diffusion Process
  10. Reverse Denoising Process
  11. Training Objective
  12. Inference
  13. Experiments
  14. Experimental Setups
  15. Overall Performance
  16. ...and 18 more sections

Figures (4)

  • Figure 1: Illustration of entity embedding distributions learned by Diffusion-based TKG reasoning framework. (a) Neglecting the negative item distribution leads to predicted entities potentially drifting toward non-interacted negative entities. (b) Incorporating the negative sampling sharpens the predictive distribution, positioning the predicted entity centroid closer to the true target.
  • Figure 2: Overview of NADEx. In Forward Diffusion, Gaussian noise is applied to the future object $o$. During the reverse denoising process, NADEx incorporates both positive and negative ($o_{t}^{n-}$) samples to disentangle plausible from implausible event predictions. For negative samples, we first stack all target embeddings in the mini-batch into a single row, copy this row $n$ times to form an $n$$\times$$n$ matrix, set the diagonal to 0 to exclude each target itself, and then take the row-wise mean of the remaining entries to obtain one compact negative prototype per target.
  • Figure 3: Performance of NADEx compared with three baselines in terms of MRR and Hit@1 on ICEWS14 under different training data scale settings.
  • Figure 4: Sensitivity analysis on ICEWS14 and ICEWS18.