PRISM: Festina Lente Proactivity -- Risk-Sensitive, Uncertainty-Aware Deliberation for Proactive Agents

TL;DR

PRISM addresses proactive intervention by modeling timing as a cost-sensitive, uncertainty-aware decision with a dual-process architecture. It learns two calibrated probabilities, $p_{ ext{need}}$ and $p_{ ext{accept}}$, and gates intervention with a dynamic threshold that balances false alarms and missed opportunities, invoking costly slow reasoning only near the decision boundary. Through RDC-distilled training that aligns supervision with deployment, PRISM achieves improved precision and reduced computational burden on ProactiveBench, with substantial gains in AUDBC and F1 while maintaining high recall. The work demonstrates a practical, reproducible approach to controllable proactivity, enabling precise and efficient interventions in real-world assistive systems.

Abstract

Proactive agents must decide not only what to say but also whether and when to intervene. Many current systems rely on brittle heuristics or indiscriminate long reasoning, which offers little control over the benefit-burden tradeoff. We formulate the problem as cost-sensitive selective intervention and present PRISM, a novel framework that couples a decision-theoretic gate with a dual-process reasoning architecture. At inference time, the agent intervenes only when a calibrated probability of user acceptance exceeds a threshold derived from asymmetric costs of missed help and false alarms. Inspired by festina lente (Latin: "make haste slowly"), we gate by an acceptance-calibrated, cost-derived threshold and invoke a resource-intensive Slow mode with counterfactual checks only near the decision boundary, concentrating computation on ambiguous and high-stakes cases. Training uses gate-aligned, schema-locked distillation: a teacher running the full PRISM pipeline provides dense, executable supervision on unlabeled interaction traces, while the student learns a response policy that is explicitly decoupled from the intervention gate to enable tunable and auditable control. On ProactiveBench, PRISM reduces false alarms by 22.78% and improves F1 by 20.14% over strong baselines. These results show that principled decision-theoretic gating, paired with selective slow reasoning and aligned distillation, yields proactive agents that are precise, computationally efficient, and controllable. To facilitate reproducibility, we release our code, models, and resources at https://prism-festinalente.github.io/; all experiments use the open-source ProactiveBench benchmark.

Figures (6)

• Figure 1: PRISM reduces false alarms by estimating confidence before speaking. The agent first estimates calibrated $(\hat{p}_{\mathrm{need}}, \hat{p}_{\mathrm{accept}})$, compares $\hat{p}_{\mathrm{accept}}$ with an adaptive threshold $\tau\left(\hat{p}_{\mathrm{need}}; C_{\mathrm{FA}}, C_{\mathrm{FN}}\right)$, and triggers a single pass slow stage only within a narrow region near the threshold. In benign edit events, low $\hat{p}_{\mathrm{need}}$ raises $\tau$, the gate remains silent, and the user continues without interruption.
• Figure 2: PRISM Pipeline. A model first estimates two calibrated probabilities from the context $X_t$: the user's need for assistance ($p_{\mathrm{need}}$) and the likely acceptance of an offer ($p_{\mathrm{accept}}$). A cost-sensitive gate then decides to intervene only if the acceptance probability meets a dynamic threshold $\tau(p_{\mathrm{need}})$ that accounts for the relative costs of false alarms ($C_{\mathrm{FA}}$) and missed opportunities ($C_{\mathrm{FN}}$). To balance accuracy and efficiency, a resource-intensive slow reasoning pass is triggered only when the initial prediction is ambiguous and falls within a narrow margin $\delta_{\mathrm{slow}}$ of the decision boundary.
• Figure 3: Efficiency-Quality Pareto Frontier. We plot AUDBC against P95 Latency across various cost ratios ($C_{\mathrm{FA}}:C_{\mathrm{FN}}$) and slow-margins ($\delta$). The blue line traces the optimal frontier, showing that PRISM (particularly at $\delta=0.1$) achieves maximal decision benefit with minimal latency overhead ($\sim$20ms). Grey points represent suboptimal configurations (e.g., higher $\delta$ or uncalibrated baselines) that incur higher latency without corresponding utility gains.
• Figure 4: Structured trace motivating PRISM. A ProactiveBench coding event (top) yields a structured output from a reasoning-capable teacher (DeepSeek-R1; middle). PRISM (bottom) abstracts the teacher’s implicit hints into calibrated signals $p_{\text{need}}$ and $p_{\text{accept}}$, uses a lightweight gate to trigger interventions when $p_{\text{need}}$ is confident, and invokes slow-mode deliberation only near uncertain margins; the final offer is regulated by $p_{\text{accept}}$. Only structured fields are analyzed; no free-form chain-of-thought is stored or surfaced.
• Figure 5: Impact of Slow Margin. Increasing the margin initially boosts performance by correcting hard examples, but latency grows linearly. $\delta=0.1$ is the "sweet spot".

Theorems & Definitions (2)

• Proposition 1: Bayes-optimal gate under asymmetric costs
• proof