TimelineKGQA: A Comprehensive Question-Answer Pair Generator for Temporal Knowledge Graphs

TL;DR

The paper addresses the scarcity and fragmentation of QA data for temporal knowledge graphs by introducing a comprehensive, multi-dimensional categorization framework and TimelineKGQA, a universal QA-pair generator with an open-source Python implementation. It defines facts as $f = (e_1, r, e_2, t_{start}, t_{end})$ and models question answering through four temporal capabilities (TCR, TPR, TSO, TAO) and four operations, enabling generation of Simple, Medium, and Complex questions with varied answer formats. Two benchmark datasets are produced from ICEWS Time Range and CronQuestion Time Point, totaling 89,372 and 41,720 questions, respectively, and evaluated with a Retrieval Augmented Generation baseline to validate that observed difficulty tracks the proposed taxonomy. The approach promises to advance TKGQA by providing diverse, scalable data for both public and private domains and by facilitating more robust temporal reasoning in downstream systems.

Abstract

Question answering over temporal knowledge graphs (TKGs) is crucial for understanding evolving facts and relationships, yet its development is hindered by limited datasets and difficulties in generating custom QA pairs. We propose a novel categorization framework based on timeline-context relationships, along with \textbf{TimelineKGQA}, a universal temporal QA generator applicable to any TKGs. The code is available at: \url{https://github.com/PascalSun/TimelineKGQA} as an open source Python package.

Figures (3)

• Figure 1: Categorization framework for Temporal Questions
• Figure 2: Question answer pair generation procedure in our TimelineKGQA Generator
• Figure 3: Temporal knowledge graph unification