Learning Multi-graph Structure for Temporal Knowledge Graph Reasoning

TL;DR

This work tackles extrapolative reasoning in Temporal Knowledge Graphs by proposing Learning Multi-graph Structure (LMS), a modular framework combining Evolutional Graph Learning, Union Graph Learning, and Temporal Graph Learning to capture concurrent, cross-timestamp, and timestamp-semantic dependencies. It introduces an adaptive gate to fuse evolutional and union representations, a timestamp-aware temporal graph, and an indicator to narrow the prediction space, all integrated with time-aware ConvTransE decoders. The approach achieves state-of-the-art results on five event-based benchmarks, significantly improving time-aware filtered and raw MRR scores, and is supported by thorough ablations and hyper-parameter analyses demonstrating the contributions of each component. The study advances TKG extrapolation by advocating a multi-graph perspective that jointly models local evolution, query-specific cross-timestamp relations, and temporal periodicities, with practical implications for more accurate future event forecasting.

Abstract

Temporal Knowledge Graph (TKG) reasoning that forecasts future events based on historical snapshots distributed over timestamps is denoted as extrapolation and has gained significant attention. Owing to its extreme versatility and variation in spatial and temporal correlations, TKG reasoning presents a challenging task, demanding efficient capture of concurrent structures and evolutional interactions among facts. While existing methods have made strides in this direction, they still fall short of harnessing the diverse forms of intrinsic expressive semantics of TKGs, which encompass entity correlations across multiple timestamps and periodicity of temporal information. This limitation constrains their ability to thoroughly reflect historical dependencies and future trends. In response to these drawbacks, this paper proposes an innovative reasoning approach that focuses on Learning Multi-graph Structure (LMS). Concretely, it comprises three distinct modules concentrating on multiple aspects of graph structure knowledge within TKGs, including concurrent and evolutional patterns along timestamps, query-specific correlations across timestamps, and semantic dependencies of timestamps, which capture TKG features from various perspectives. Besides, LMS incorporates an adaptive gate for merging entity representations both along and across timestamps effectively. Moreover, it integrates timestamp semantics into graph attention calculations and time-aware decoders, in order to impose temporal constraints on events and narrow down prediction scopes with historical statistics. Extensive experimental results on five event-based benchmark datasets demonstrate that LMS outperforms state-of-the-art extrapolation models, indicating the superiority of modeling a multi-graph perspective for TKG reasoning.

Figures (5)

• Figure 1: An illustration of extrapolation over TKGs.
• Figure 2: An illustration of $\mathsf{LMS}$ model architecture. Firstly, a series of TKG sequences are learned through Evolutional Graph Learning (§ \ref{['egl']}). Then, Union Graph Learning (§ \ref{['ugl']}) constructs a query-specific graph based on the sequence and queries and captures the structural interactions among those facts, while temporal information (§ \ref{['tgl']}) is combined in this step. After that, the Time-aware Decoder (§ \ref{['dec']}) is adopted to predict future facts according to Indicator (§ \ref{['ind']}) and the combined features from Adaptive Gate (§ \ref{['gate']}).
• Figure 3: An illustration of the union graph construction.
• Figure 4: TKG entity extrapolation results in comparison with HGLS under raw setting on ICEWS14s, ICEWS05-15, ICEWS18, and GDELT.
• Figure 5: Analysis of hyper-parameter $k$ and $\alpha$ on ICEWS14s.