Combining Cost-Constrained Runtime Monitors for AI Safety

TL;DR

This work tackles cost‑aware runtime monitoring for AI safety by formulating the problem as maximizing recall $P(A=1|B=1)$ under a unified budget $Y$ that covers both monitoring and auditing. It decomposes the policy into monitor invocation and audit allocation, and uses the Neyman–Pearson lemma to guide intervention based on likelihood ratios derived from monitor scores, with an exhaustive search over simple policy classes. Empirical results in backdoored code detection show that likelihood‑ratio based policies outperform naive baselines, and that combining monitors with different costs can yield Pareto improvements across budgets. The framework provides a principled, actionable method for resource allocation in safety monitoring and points to future work on robustness to distribution shifts and joint monitor optimization.

Abstract

Monitoring AIs at runtime can help us detect and stop harmful actions. In this paper, we study how to efficiently combine multiple runtime monitors into a single monitoring protocol. The protocol's objective is to maximize the probability of applying a safety intervention on misaligned outputs (i.e., maximize recall). Since running monitors and applying safety interventions are costly, the protocol also needs to adhere to an average-case budget constraint. Taking the monitors' performance and cost as given, we develop an algorithm to find the best protocol. The algorithm exhaustively searches over when and which monitors to call, and allocates safety interventions based on the Neyman-Pearson lemma. By focusing on likelihood ratios and strategically trading off spending on monitors against spending on interventions, we more than double our recall rate compared to a naive baseline in a code review setting. We also show that combining two monitors can Pareto dominate using either monitor alone. Our framework provides a principled methodology for combining existing monitors to detect undesirable behavior in cost-sensitive settings.

Figures (10)

• Figure 1: Combining both monitors can achieve higher safety at the same cost. See Figure \ref{['llr_2d']} for an illustration of how we use two monitors and Appendix Section \ref{['relative_perf']} for standard errors on relative performance as opposed to absolute performance.
• Figure 2: We see in subplot (a) that there is a dip in the log likelihood ratio after around 7.2. The optimal policy in the $\Pi_{\text{NP}}$ class audits scores around 7.2 first instead of higher scores. It achieves higher safety as a result (b). See Figure \ref{['onemon_hist']} in the appendix for a histogram of monitor scores. Since both policies always use the monitor and there's only one monitor, we convert our unified budget to just the false positive rate (i.e., $(Y - c_1)/c_a$) when discussing their performance.
• Figure 3: $\Pi_{\text{NPS}}$ policies outperform by using the monitors less and auditing more.
• Figure 4: Panel (a) shows the underlying distribution of monitor scores, and panel (b) shows an optimal policy with colors indicating the estimated log likelihood ratio at each point.
• Figure 5: $\Pi_{\text{SR}}$ beats single-monitor and two-monitor baselines.