Confidence over Time: Confidence Calibration with Temporal Logic for Large Language Model Reasoning

TL;DR

We address the mismatch between long-form reasoning in LLMs and confidence estimation by modeling confidence as a stepwise signal $\mathbf S=[s_1,\dots,s_n]$ derived from token probabilities, where $s_j=\frac{1}{L_j}\sum_{k=0}^{L_j-1}P(y_{t_j+k}\mid \mathbf{y}_{<t_j+k},\mathbf{x})$. We mine STL formulas $\varphi_{\boldsymbol{\theta}}$ whose robustness $\rho(\varphi_{\boldsymbol{\theta}},\mathbf{S})$ discriminates correct vs incorrect responses, yielding dual sets $\Phi_{\text{pos}}$ and $\Phi_{\text{neg}}$. To keep interpretability, we fix the STL structure and use a hypernetwork $\mathcal{H}_{\psi}$ to predict $\boldsymbol{\theta}$ from $(\boldsymbol{x},\mathbf{S})$, mapping robustness through $\hat p=\omega(\alpha\rho+\beta)$. Empirically, across GAOKAO-Math, CLadder, SciQ, BBH and backbones $\text{Qwen3-8B}$, $\text{Gemma-3-12B}$, $\text{Llama-3-8B}$, negative-pattern templates transfer across tasks, while in-domain mining with instance-adaptive parameters delivers the most reliable calibration (ECE, BS, AUROC) with single-sample inference around $0.55_{\pm 0.04}$ seconds per example.

Abstract

Large Language Models (LLMs) increasingly rely on long-form, multi-step reasoning to solve complex tasks such as mathematical problem solving and scientific question answering. Despite strong performance, existing confidence estimation methods typically reduce an entire reasoning process to a single scalar score, ignoring how confidence evolves throughout the generation. As a result, these methods are often sensitive to superficial factors such as response length or verbosity, and struggle to distinguish correct reasoning from confidently stated errors. We propose to characterize the stepwise confidence signal using Signal Temporal Logic (STL). Using a discriminative STL mining procedure, we discover temporal formulas that distinguish confidence signals of correct and incorrect responses. Our analysis found that the STL patterns generalize across tasks, and numeric parameters exhibit sensitivity to individual questions. Based on these insights, we develop a confidence estimation approach that informs STL blocks with parameter hypernetworks. Experiments on multiple reasoning tasks show our confidence scores are more calibrated than the baselines.

Figures (12)

• Figure 1: Overview of stepwise confidence modeling with STL. Token-level probabilities are aggregated into a stepwise confidence signal $\mathbf{S}$, on which STL formulas are evaluated to capture temporal confidence patterns via robustness scores.
• Figure 2: Conceptual illustration of Research Question 1 (RQ1) and Research Question 2 (RQ2). RQ1 examines whether the STL structures mined from different task types are shared or task-specific. RQ2 studies, under a fixed STL structure, whether the associated parameters $\theta$ learned during optimization generalize across questions.
• Figure 3: Pairwise similarity of mined STL formulas across four reasoning benchmarks. Left: similarity of STL patterns characterizing correct reasoning. Right: similarity of STL patterns characterizing incorrect reasoning. Similarity is measured using the Jaccard index over mined formula sets.
• Figure 4: RQ2: Pairwise differences of optimized STL parameters across individual questions under a fixed STL structure. Parameters are grouped into three categories: predicate thresholds, difference-related parameters, and time-related parameters. Results are shown for BBH and CLadder using Qwen3-8B.
• Figure 5: Overview of the confidence quantification pipeline using STL blocks. During training, discriminative STL mining identifies temporal formula structures, which are instantiated as STL blocks with learnable parameters. At test time, the learned STL structures are fixed and applied to stepwise confidence signals to produce a scalar confidence score.