When Silence Is Golden: Can LLMs Learn to Abstain in Temporal QA and Beyond?

TL;DR

This work investigates learning abstention for temporal question answering with LLMs. It introduces a reinforcement learning framework (GRPO) that combines CoT supervision with abstention-aware rewards to train models, including explicit CoT data and implicit cues like time filtering and temporal knowledge graphs. The results show that a 1.5B model trained with CoT-SFT and RL can surpass GPT-4o on TimeQA in both easy and hard settings, while SFT alone tends to induce overconfidence. However, abstention generalizes poorly to out-of-domain tasks, highlighting the need for principled uncertainty-aware training and robust evaluation for reliable abstention across domains.

Abstract

Large language models (LLMs) rarely admit uncertainty, often producing fluent but misleading answers, rather than abstaining (i.e., refusing to answer). This weakness is even evident in temporal question answering, where models frequently ignore time-sensitive evidence and conflate facts across different time-periods. In this paper, we present the first empirical study of training LLMs with an abstention ability while reasoning about temporal QA. Existing approaches such as calibration might be unreliable in capturing uncertainty in complex reasoning. We instead frame abstention as a teachable skill and introduce a pipeline that couples Chain-of-Thought (CoT) supervision with Reinforcement Learning (RL) guided by abstention-aware rewards. Our goal is to systematically analyze how different information types and training techniques affect temporal reasoning with abstention behavior in LLMs. Through extensive experiments studying various methods, we find that RL yields strong empirical gains on reasoning: a model initialized by Qwen2.5-1.5B-Instruct surpasses GPT-4o by $3.46\%$ and $5.80\%$ in Exact Match on TimeQA-Easy and Hard, respectively. Moreover, it improves the True Positive rate on unanswerable questions by $20\%$ over a pure supervised fine-tuned (SFT) variant. Beyond performance, our analysis shows that SFT induces overconfidence and harms reliability, while RL improves prediction accuracy but exhibits similar risks. Finally, by comparing implicit reasoning cues (e.g., original context, temporal sub-context, knowledge graphs) with explicit CoT supervision, we find that implicit information provides limited benefit for reasoning with abstention. Our study provides new insights into how abstention and reasoning can be jointly optimized, providing a foundation for building more reliable LLMs.

Figures (6)

• Figure 1: Qualitative and quantitative illustrations of LLM's abstention ability. Both demonstrate that temporal abstention is still challenging for LLMs.
• Figure 2: Overview of the CoT+RL pipeline. High-quality CoT reasoning data are first generated and filtered to form a trusted reasoning set, which is used for SFT. The SFT model is then further optimized with RL to enhance reasoning and abstention capabilities using format and answer reward.
• Figure 3: Abstention performance comparisons across various prompts on TimeQA task with Qwen2.5-1.5B-Instruct and original context ${\bm{c}}$. Van., Con., Pos., and Neg. denote for Vanilla, Contrastive, Positive, Negative Prompts, respectively.
• Figure 4: Abstention performance comparisons across different numbers of KGs (a) and different training methods (b).
• Figure 5: The KG-related information extraction pipeline. Knowledge Graphs are extracted by GPT-4o-mini given the contexts, which are ranked in either a semantic-based or a lexical-based way.