Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning

TL;DR

This work tackles interpretability and adaptability in Temporal Knowledge Graph Reasoning by introducing LLM-DA, a framework that uses large language models to generate temporal logical rules from historical data and then dynamically adapt those rules with the latest events without fine-tuning the LLMs. A contextual relation selector narrows input context to the most relevant relations, while a constrained Markovian sampling process extracts temporal patterns and rules. Dynamic Adaptation continuously updates the LLM-generated rules using current data, aided by a confidence-based pruning mechanism, and a rule- and graph-based candidate reasoning module combines symbolic and neural signals to predict future facts. Experiments on ICEWS datasets show that LLM-DA improves link prediction accuracy over strong baselines, including other LLM-based TKGR methods, and ablations demonstrate the value of combining historical and current data as well as iterative adaptation. The approach offers a view into interpretable, updatable TKGR and highlights practical directions for reducing reliance on expensive LLM fine-tuning while maintaining performance on evolving knowledge graphs.

Abstract

Temporal Knowledge Graph Reasoning (TKGR) is the process of utilizing temporal information to capture complex relations within a Temporal Knowledge Graph (TKG) to infer new knowledge. Conventional methods in TKGR typically depend on deep learning algorithms or temporal logical rules. However, deep learning-based TKGRs often lack interpretability, whereas rule-based TKGRs struggle to effectively learn temporal rules that capture temporal patterns. Recently, Large Language Models (LLMs) have demonstrated extensive knowledge and remarkable proficiency in temporal reasoning. Consequently, the employment of LLMs for Temporal Knowledge Graph Reasoning (TKGR) has sparked increasing interest among researchers. Nonetheless, LLMs are known to function as black boxes, making it challenging to comprehend their reasoning process. Additionally, due to the resource-intensive nature of fine-tuning, promptly updating LLMs to integrate evolving knowledge within TKGs for reasoning is impractical. To address these challenges, in this paper, we propose a Large Language Models-guided Dynamic Adaptation (LLM-DA) method for reasoning on TKGs. Specifically, LLM-DA harnesses the capabilities of LLMs to analyze historical data and extract temporal logical rules. These rules unveil temporal patterns and facilitate interpretable reasoning. To account for the evolving nature of TKGs, a dynamic adaptation strategy is proposed to update the LLM-generated rules with the latest events. This ensures that the extracted rules always incorporate the most recent knowledge and better generalize to the predictions on future events. Experimental results show that without the need of fine-tuning, LLM-DA significantly improves the accuracy of reasoning over several common datasets, providing a robust framework for TKGR tasks.

Figures (8)

• Figure 1: A brief description of LLM-DA. Specifically, LLM-DA harnesses LLMs to formulate general rules on historical data. Subsequently, LLM-DA dynamically guides the LLMs to update these rules based on current data, ensuring they more accurately reflect the objective distribution.
• Figure 2: The Framework of LLM-DA. Specifically, LLM-DA first analyzes historical data to extract temporal rules and utilizes the powerful generative capabilities of LLMs to generate general rules. Subsequently, LLM-DA updates these rules using current data. Finally, the updated rules are applied to predict future events.
• Figure 3: The constraints of the constrained Markovian random walks. "✗" denotes this path does not exist, "$P_i$" indicates the transition probability.
• Figure 4: Time interval segmented prediction: MRR performance on both datasets compared to different baselines.
• Figure 5: Comparison with different number of iterations on both datasets.