AMCEN: An Attention Masking-based Contrastive Event Network for Two-stage Temporal Knowledge Graph Reasoning

TL;DR

AMCEN tackles the imbalance between recurring and new events in temporal knowledge graphs by introducing a two-stage reasoning framework that uses historical and non-historical attention masks together with a local-global temporal encoder. The model combines a CompGCN-based structural encoder, a local-global temporal encoder, attention-masking decoders, and a contrastive event classifier to refine future-event predictions. Empirical results on four benchmarks show consistent improvements, with ablations validating the contribution of attention masking and the two-stage strategy. The work advances extrapolative temporal reasoning by explicitly separating the exploration of historical versus non-historical entities and integrating local-global temporal cues into contrastive learning for sharper classification and decoding.

Abstract

Temporal knowledge graphs (TKGs) can effectively model the ever-evolving nature of real-world knowledge, and their completeness and enhancement can be achieved by reasoning new events from existing ones. However, reasoning accuracy is adversely impacted due to an imbalance between new and recurring events in the datasets. To achieve more accurate TKG reasoning, we propose an attention masking-based contrastive event network (AMCEN) with local-global temporal patterns for the two-stage prediction of future events. In the network, historical and non-historical attention mask vectors are designed to control the attention bias towards historical and non-historical entities, acting as the key to alleviating the imbalance. A local-global message-passing module is proposed to comprehensively consider and capture multi-hop structural dependencies and local-global temporal evolution for the in-depth exploration of latent impact factors of different event types. A contrastive event classifier is used to classify events more accurately by incorporating local-global temporal patterns into contrastive learning. Therefore, AMCEN refines the prediction scope with the results of the contrastive event classification, followed by utilizing attention masking-based decoders to finalize the specific outcomes. The results of our experiments on four benchmark datasets highlight the superiority of AMCEN. Especially, the considerable improvements in Hits@1 prove that AMCEN can make more precise predictions about future occurrences.

Figures (5)

• Figure 1: The proportion of recurring events in different datasets.
• Figure 2: An illustration of a two-stage reasoning process.
• Figure 3: The overall architecture of AMCEN.
• Figure 4: The distribution of new events in different datasets.
• Figure 5: Sensitivity analysis results of hyperparameters in MRR