OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows

TL;DR

This work tackles the safety of autonomous mobile GUI agents by introducing MobileRisk-Live, a dynamic Android-emulator sandbox, and MobileRisk, a benchmark of fine-grained, annotated trajectories. It then proposes OS-Sentinel, a hybrid safety detector that couples a Formal Verifier for deterministic system-level checks with a Contextual Judge powered by Vision-Language Models for context-aware analysis. Through extensive experiments across multiple backbones and environments, OS-Sentinel consistently outperforms rule-based and purely model-based baselines at both trajectory and step levels, while maintaining practical latency. The study provides a robust foundation for safe, reliable mobile GUI automation and offers insights into component contributions, taxonomy coverage, and reproducible evaluation in realistic settings.

Abstract

Computer-using agents powered by Vision-Language Models (VLMs) have demonstrated human-like capabilities in operating digital environments like mobile platforms. While these agents hold great promise for advancing digital automation, their potential for unsafe operations, such as system compromise and privacy leakage, is raising significant concerns. Detecting these safety concerns across the vast and complex operational space of mobile environments presents a formidable challenge that remains critically underexplored. To establish a foundation for mobile agent safety research, we introduce MobileRisk-Live, a dynamic sandbox environment accompanied by a safety detection benchmark comprising realistic trajectories with fine-grained annotations. Built upon this, we propose OS-Sentinel, a novel hybrid safety detection framework that synergistically combines a Formal Verifier for detecting explicit system-level violations with a VLM-based Contextual Judge for assessing contextual risks and agent actions. Experiments show that OS-Sentinel achieves 10%-30% improvements over existing approaches across multiple metrics. Further analysis provides critical insights that foster the development of safer and more reliable autonomous mobile agents. Our code and data are available at https://github.com/OS-Copilot/OS-Sentinel.

Figures (13)

• Figure 1: A normal user request can trigger unexpected safety issues in mobile agents, such as privacy violations and socially offensive behaviors.
• Figure 2: MobileRisk-Live provides a dynamic Android emulator sandbox that supports realistic GUI agent interactions and state restoration. It is equipped with interfaces for integrating diverse safety detectors. MobileRisk is constructed from frozen trajectories collected in this environment, where each step contains a screenshot, a11ytree, system state trace, and action. These trajectories are further annotated at multiple granularities.
• Figure 3: Construction pipeline of MobileRisk, where raw instructions are executed by agents to produce trajectories, which are then inspected, refined, and labeled.
• Figure 4: Illustration of OS-Sentinel, a hybrid safety detection framework. At each step, agent actions, GUI observations, and system state traces are jointly assessed by a contextual judge and a formal verifier to determine safety. The framework supports safety detection for mobile GUI agents at both the step level and the trajectory level.
• Figure 5: Trajectory-level component analysis across three backbones (Accuracy).