MirrorGuard: Toward Secure Computer-Use Agents via Simulation-to-Real Reasoning Correction

TL;DR

MirrorGuard tackles the safety-utility dilemma of autonomous computer-use agents by shifting defense from blocking actions to correcting the agent’s reasoning at the planning stage. It trains a compact vision-language corrector in a text-based neural-symbolic MirrorWorld, enforcing object-permanence and causal reasoning to learn safe intent before execution, and then transfers these safety policies to real GUI contexts via cross-modal latent alignment. The approach yields substantial reductions in unsafe behavior across diverse CUAs and benchmarks, with a low false-refusal rate that preserves task progress, achieving a practical sim-to-real defense. The work demonstrates that safety principles learned in a symbolic, modality-agnostic space can generalize to multimodal, high-fidelity GUI environments, offering a scalable path to robust real-world protection for CUAs. Overall, MirrorGuard provides a plug-and-play defense that improves security without sacrificing agent utility, highlighting the value of neural-symbolic simulation and cross-modal transfer in AI safety for system-level agents.

Abstract

Large foundation models are integrated into Computer Use Agents (CUAs), enabling autonomous interaction with operating systems through graphical user interfaces (GUIs) to perform complex tasks. This autonomy introduces serious security risks: malicious instructions or visual prompt injections can trigger unsafe reasoning and cause harmful system-level actions. Existing defenses, such as detection-based blocking, prevent damage but often abort tasks prematurely, reducing agent utility. In this paper, we present MirrorGuard, a plug-and-play defense framework that uses simulation-based training to improve CUA security in the real world. To reduce the cost of large-scale training in operating systems, we propose a novel neural-symbolic simulation pipeline, which generates realistic, high-risk GUI interaction trajectories entirely in a text-based simulated environment, which captures unsafe reasoning patterns and potential system hazards without executing real operations. In the simulation environment, MirrorGuard learns to intercept and rectify insecure reasoning chains of CUAs before they produce and execute unsafe actions. In real-world testing, extensive evaluations across diverse benchmarks and CUA architectures show that MirrorGuard significantly mitigates security risks. For instance, on the ByteDance UI-TARS system, it reduces the unsafe rate from 66.5% to 13.0% while maintaining a marginal false refusal rate (FRR). In contrast, the state-of-the-art GuardAgent only achieves a reduction to 53.9% and suffers from a 15.4% higher FRR. Our work proves that simulation-derived defenses can provide robust, real-world protection while maintaining the fundamental utility of the agent. Our code and model are publicly available at https://bmz-q-q.github.io/MirrorGuard/.

Figures (9)

• Figure 1: Overview of MirrorGuard: "Train in the MirrorWorld, Act in the Wild."MirrorGuard trains a reasoning correction module entirely within a text-based MirrorWorld, where a ReAct agent interacts with a symbolic environment and insecure thoughts are automatically identified and corrected under explicit security rules. These paired trajectories are used for supervised fine-tuning to learn safety-aware reasoning. At deployment, the trained MirrorGuard intercepts and corrects unsafe reasoning in real-world GUI environments before execution, enabling scalable sim-to-real transfer of security reasoning while avoiding irreversible system damage and preserving agent utility.
• Figure 2: Key vulnerabilities of computer use agents: (1) Visual semantic spoofing, (2) multi-modal reasoning collapse, and (3) implicit privilege escalation.
• Figure 3: Overview of the MirrorWorld Simulation Pipeline. (1) Task Creation: High-stealth risk scenarios (e.g., disinformation creation using GIMP) are synthesized from seed applications via a hierarchical generation process. (2) Neural–Symbolic Simulator: A hybrid environment in which the agent interacts with a structured WorldState, producing grounded trajectories while avoiding real OS-level execution.
• Figure 4: Overview of the MirrorGuard workflow. The pipeline consists of two phases: (1) Training, where the model learns to map insecure reasoning to secure templates using text-based simulation data generated from the MirrorWorld (Section \ref{['sec:mirror_world']}); and (2) Deployment, where the fine-tuned VLM leverages its pre-aligned visual-textual latent space to identify and correct risky thoughts in real-time GUI environments, ensuring the generation of secure actions.
• Figure 5: Comparison of the safety rates of target agents across different task types on RiOSWorld.