Temp-R1: A Unified Autonomous Agent for Complex Temporal KGQA via Reverse Curriculum Reinforcement Learning

TL;DR

Temp-R1 tackles the challenge of Temporal Knowledge Graph Question Answering by replacing fixed-workflow pipelines with an autonomous end-to-end agent. It introduces an expanded internal action space (<plan>, <filter>, <rank>) alongside an external <search> tool and trains via a two-stage process: supervised fine-tuning for structured reasoning followed by Group Relative Policy Optimization with reverse curriculum learning to avoid shortcut traps. The approach achieves state-of-the-art results on MultiTQ and TimelineKGQA with an 8B open-source backbone, while significantly reducing inference costs by avoiding reliance on closed-source APIs. Ablation studies confirm the importance of internal reasoning actions, SFT cold-start, and reverse curriculum learning for robust temporal reasoning and generalization across domains. This work proposes a new paradigm for autonomous temporal reasoning agents in TKQA and demonstrates scalable, cost-efficient deployment potential.

Abstract

Temporal Knowledge Graph Question Answering (TKGQA) is inherently challenging, as it requires sophisticated reasoning over dynamic facts with multi-hop dependencies and complex temporal constraints. Existing methods rely on fixed workflows and expensive closed-source APIs, limiting flexibility and scalability. We propose Temp-R1, the first autonomous end-to-end agent for TKGQA trained through reinforcement learning. To address cognitive overload in single-action reasoning, we expand the action space with specialized internal actions alongside external action. To prevent shortcut learning on simple questions, we introduce reverse curriculum learning that trains on difficult questions first, forcing the development of sophisticated reasoning before transferring to easier cases. Our 8B-parameter Temp-R1 achieves state-of-the-art performance on MultiTQ and TimelineKGQA, improving 19.8% over strong baselines on complex questions. Our work establishes a new paradigm for autonomous temporal reasoning agents. Our code will be publicly available soon at https://github.com/zjukg/Temp-R1.

Figures (10)

• Figure 1: Paradigm shift in TKGQA systems. Existing approaches (a) suffer from fixed workflow and high API cost. Temp-R1 (b) is an autonomous end-to-end agent trained for dynamic temporal reasoning.
• Figure 2: Comparison of internal reasoning mechanisms: Temp-R1 vs. Search-R1. By decoupling internal reasoning into explicit <filter> and <rank> actions (right), Temp-R1 maintains logical rigor and eliminates hallucinations in temporal sequencing. In contrast, the monolithic <think> block in Search-R1 (left) suffers from cognitive overload, leading to a failure in processing the retrieved temporal facts.
• Figure 3: Overall architecture of Temp-R1. The rollout loop features an expanded internal and external action space ($\mathcal{A}_{internal}$ and $\mathcal{A}_{external}$) designed for structured temporal reasoning. The model is trained via a two-stage process: supervised cold start from high-quality trajectories, followed by GRPO-based reinforcement learning supported by reverse curriculum learning strategies.
• Figure 4: Performance comparison of Temp-R1 across different backbone models (GPT-4o, GPT-4o-mini, LLaMA3.1-8B-it, Qwen2.5 series) and training stages (base model, w/SFT cold start, w/GRPO training).
• Figure 5: Training dynamics across different Qwen2.5 backbone sizes (7B, 3B, 1.5B). (a) Training Reward, (b) Policy Entropy, and (c) Val Accuracy