From Refusal to Recovery: A Control-Theoretic Approach to Generative AI Guardrails

TL;DR

The paper reframes generative AI safety as a sequential decision problem where downstream harms emerge from continual agent–world interactions. It introduces predictive guardrails that operate in an LLM's latent representation, combining a real-time safety monitor with a recovery policy rooted in safety-critical control theory, and demonstrates model-agnostic applicability. A scalable safety-centric reinforcement learning recipe is proposed to approximate the optimal safety value $V^*(z^{AI})$ and policy $π^{AI}_*$, enabling synthesis of guardrails that work across driving, commerce, and assistive tasks. Across simulated domains, the guardrails improve safety outcomes and maintain task performance, offering a principled alternative to traditional refusal-based guardrails with broad practical impact for embodied and digital AI agents.

Abstract

Generative AI systems are increasingly assisting and acting on behalf of end users in practical settings, from digital shopping assistants to next-generation autonomous cars. In this context, safety is no longer about blocking harmful content, but about preempting downstream hazards like financial or physical harm. Yet, most AI guardrails continue to rely on output classification based on labeled datasets and human-specified criteria,making them brittle to new hazardous situations. Even when unsafe conditions are flagged, this detection offers no path to recovery: typically, the AI system simply refuses to act--which is not always a safe choice. In this work, we argue that agentic AI safety is fundamentally a sequential decision problem: harmful outcomes arise from the AI system's continually evolving interactions and their downstream consequences on the world. We formalize this through the lens of safety-critical control theory, but within the AI model's latent representation of the world. This enables us to build predictive guardrails that (i) monitor an AI system's outputs (actions) in real time and (ii) proactively correct risky outputs to safe ones, all in a model-agnostic manner so the same guardrail can be wrapped around any AI model. We also offer a practical training recipe for computing such guardrails at scale via safety-critical reinforcement learning. Our experiments in simulated driving and e-commerce settings demonstrate that control-theoretic guardrails can reliably steer LLM agents clear of catastrophic outcomes (from collisions to bankruptcy) while preserving task performance, offering a principled dynamic alternative to today's flag-and-block guardrails.

Figures (6)

• Figure 1: Overview of Control-Theoretic Predictive Guardrails for AI Agents. This guardrail, which shields an LLM agent, operates on text-based observations of the world, but learns a latent-space predictive safety monitor and recovery policy from real-world signals reflecting the outcomes of its actions (e.g., obstacle collisions).
• Figure 2: Environments. Three LLM agent environments where we evaluate our guardrails.
• Figure 3: Agentic Commerce. One rollout of the base LLM agent vs. ReGuard vs. GPT-4o.
• Figure 4: Agentic Driving: Shielding. Success rate, failure rate, rollout f1 score, intervention rates.
• Figure 5: Backseat Driving: Shielding. Shielded rollouts of ReGuard influencing the none and urgent human proxy personas to stay safe.