Multi-hop Question Answering under Temporal Knowledge Editing

TL;DR

This work tackles multi-hop question answering under temporal knowledge editing by introducing TEMPLE-MQA, which constructs a Time-Aware Graph (TAG) of edits and uses an inference-path planner plus a structural retrieval mechanism to reason over temporal knowledge. The framework combines LLM-driven path planning with joint reasoning over the TAG, enabling effective handling of ripple effects and explicit temporal contexts without retraining the model. It also introduces TKEMQA, a dedicated benchmark for MQA with temporal scopes, and demonstrates substantial performance gains over strong baselines across several datasets, along with insights from ablations and robustness analyses. The proposed approach offers a practical and scalable solution for timely knowledge updates in complex, multi-hop reasoning tasks with explicit temporal constraints.

Abstract

Multi-hop question answering (MQA) under knowledge editing (KE) has garnered significant attention in the era of large language models. However, existing models for MQA under KE exhibit poor performance when dealing with questions containing explicit temporal contexts. To address this limitation, we propose a novel framework, namely TEMPoral knowLEdge augmented Multi-hop Question Answering (TEMPLE-MQA). Unlike previous methods, TEMPLE-MQA first constructs a time-aware graph (TAG) to store edit knowledge in a structured manner. Then, through our proposed inference path, structural retrieval, and joint reasoning stages, TEMPLE-MQA effectively discerns temporal contexts within the question query. Experiments on benchmark datasets demonstrate that TEMPLE-MQA significantly outperforms baseline models. Additionally, we contribute a new dataset, namely TKEMQA, which serves as the inaugural benchmark tailored specifically for MQA with temporal scopes.

Figures (5)

• Figure 1: Limitations of dense retrieval to handle temporal context. (a) There is a temporal constraint in question. However, the dense retrieval mechanism fails to accurately capture this temporal information, thus yielding an erroneous retrieval. (b) Our time-aware graph stores edits as a time-conscious knowledge structure to help distinguish temporal context and improve retrieval accuracy.
• Figure 2: Overview of Temple-MQA. Given a multi-hop question, Temple-MQA employs LLMs to strategize an inference path for the question. Then, it leverages LLMs and the TAG ($\mathcal{G}_{t}$) for joint reasoning on the inference path.
• Figure 3: The workflow of structural retrieval. 1) The first step is to extract the 1-hop sub-graph where the center point is Tom. We filter out the edits not conforming with the temporal scope of the query. 2) The second step calculates the semantic similarity of the relation and concept between the query and knowledeg in sub-graph.
• Figure 4: The line graph depicting the decrease in M-Acc with the increase in edit batch size on datasets MQuAKE-CF-3K, TKeMqa and MQuAKE-CF-3K-Fix.
• Figure 5: A bar graph comparing the M-Acc of three methods under two settings: All-edited for Temple-MQA and 1-edited for MQuAKE-CF-3K.