Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion

TL;DR

This work tackles TKGC by enabling LLMs to reason over temporal graphs through a unified pipeline that combines structure-aware historical context, reverse-logic prompting, and parameter-efficient fine-tuning via LoRA. It introduces structure-based history augmentation (schema-matching, entity-augmented, and relation-augmented histories) and reverse-logic data to mitigate reasoning reversals, showing robust improvements across multiple TKGC datasets. Extensive ablations reveal the value of augmentation, reverse logic, and optimized history length, while larger model size yields diminishing returns and commercial LLMs show mixed results depending on dataset and setting. Overall, the approach narrows the gap to specialized TKGC models and offers a transferable, data-efficient path for leveraging LLMs in temporal knowledge inference, with implications for future integration of LLMs and structured temporal reasoning.

Abstract

Temporal Knowledge Graph Completion (TKGC) is a complex task involving the prediction of missing event links at future timestamps by leveraging established temporal structural knowledge. This paper aims to provide a comprehensive perspective on harnessing the advantages of Large Language Models (LLMs) for reasoning in temporal knowledge graphs, presenting an easily transferable pipeline. In terms of graph modality, we underscore the LLMs' prowess in discerning the structural information of pivotal nodes within the historical chain. As for the generation mode of the LLMs utilized for inference, we conduct an exhaustive exploration into the variances induced by a range of inherent factors in LLMs, with particular attention to the challenges in comprehending reverse logic. We adopt a parameter-efficient fine-tuning strategy to harmonize the LLMs with the task requirements, facilitating the learning of the key knowledge highlighted earlier. Comprehensive experiments are undertaken on several widely recognized datasets, revealing that our framework exceeds or parallels existing methods across numerous popular metrics. Additionally, we execute a substantial range of ablation experiments and draw comparisons with several advanced commercial LLMs, to investigate the crucial factors influencing LLMs' performance in structured temporal knowledge inference tasks.

Figures (2)

• Figure 1: LLM undergoes fine-tuning on known data and subsequently utilizes the chain of known factual information to generate the next event.
• Figure 2: The evolution pattern of the Hits@1 metric across four utilized datasets concerning the history length $L$.