Anytime Safe PAC Efficient Reasoning

TL;DR

B-PAC reasoning introduces an online, model-agnostic framework for anytime-safe efficient reasoning by routing queries between a high-cost thinking model and a low-cost non-thinking model. It builds an IPS-based risk estimator and a wealth process as a betting game to adapt a time-varying routing threshold, ensuring the risk of incorrect non-thinking usage stays below a user-specified tolerance with high probability, even under partial feedback and non-stationary data. The approach achieves significant efficiency gains (substantial reductions in thinking usage) while maintaining formal safety guarantees via fixed-sequence testing and martingale techniques, demonstrated across diverse benchmarks. This work advances practical deployment of large reasoning models by providing dynamically adaptive, theoretically grounded routing that scales to online and shifting data environments, with meaningful impact for real-time AI systems requiring both speed and reliability.

Abstract

Large Reasoning Models (LRMs) have demonstrated remarkable performance on complex tasks but suffer from high computational costs and latency. While selective thinking strategies improve efficiency by routing easy queries to non-thinking models, existing approaches often incur uncontrollable errors, especially in online settings where the performance loss of a non-thinking model is only partially observed and data are non-stationary. To address this, we propose Betting Probably Approximately Correct (B-PAC) reasoning, a principled method that enables anytime safe and efficient online reasoning under partial feedback. Specifically, we utilize inverse propensity scoring estimators to construct test supermartingales for candidate thresholds, and then dynamically adjust the routing threshold based on the accumulated statistical evidence of safety. Theoretically, we establish the anytime-valid performance loss control and the efficiency of B-PAC reasoning. Extensive experiments demonstrate that B-PAC reasoning significantly reduces computational overhead, decreasing thinking model usage by up to 81.01\%, while controlling the performance loss below the user-specified level.

Figures (9)

• Figure 1: Efficiency outperforms offline PAC reasoning. ER, ECP, and TP are reported on a combined dataset of Magpie and BBH, with $\epsilon=0.08$ and $\alpha=0.1$. The vertical dotted line indicates the size of the calibration set used for the offline PAC baseline. Experiments are repeated 100 times, and the shaded areas represent standard deviations.
• Figure 2: Anytime safety for non-stationary data. Results are reported on a combined dataset of MMLU-Pro and BBH, with $\epsilon=0.05$ and $\alpha=0.1$.
• Figure 3: Safety and efficiency of B-PAC reasoning compared with online methods, including IPS+Hoeff and O-Naive. Results are reported on MMLU-Pro and BBH benchmarks, with $\epsilon=0.08$ and $\alpha=0.1$.
• Figure 4: Prompt used for evaluating response quality on Magpie dataset.
• Figure 5: Performance of B-PAC reasoning under different tolerance levels. We consider $\epsilon\in\{0.05,0.06,0.07,0.08,0.0.9,0.10\}$ on four benchmark,s including Magpie, BBH, MMLU-Pro, and MATH. B-PAC reasoning ensures anytime-valid performance loss control across all settings, and achieves higher efficiency as $\epsilon$ increases.

Theorems & Definitions (24)

• Definition 2.1: Anytime $(\epsilon,\alpha)$-PAC efficiency
• Remark 2.2: Data Assumption
• Remark 2.3: Loss Function
• Remark 3.1: Uncertainty Score
• Remark 3.2: Non-trivial setting
• Lemma 4.1: Supermartingale Property
• Theorem 4.2: Anytime $(\epsilon,\alpha)$-PAC efficient
• Theorem 4.3
• Theorem 5.1: Safety
• Remark 6.1: ECP vs TP