MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control

TL;DR

MobileSafetyBench addresses the urgent need to evaluate safety for autonomous agents controlling mobile devices in realistic settings. By leveraging Android emulators and a diverse task suite that includes high-risk and indirect-prompt-injection scenarios, the benchmark reveals systemic safety weaknesses in state-of-the-art LLM agents. The authors introduce Safety-guided Chain-of-Thought prompting to bolster safety, but experiments show substantial room for improvement and vulnerabilities to indirect prompt injection. The work provides open-source benchmarks and tooling to accelerate development of safer, more trustworthy mobile-device control agents with practical implications for deployment and governance.

Abstract

Autonomous agents powered by large language models (LLMs) show promising potential in assistive tasks across various domains, including mobile device control. As these agents interact directly with personal information and device settings, ensuring their safe and reliable behavior is crucial to prevent undesirable outcomes. However, no benchmark exists for standardized evaluation of the safety of mobile device-control agents. In this work, we introduce MobileSafetyBench, a benchmark designed to evaluate the safety of device-control agents within a realistic mobile environment based on Android emulators. We develop a diverse set of tasks involving interactions with various mobile applications, including messaging and banking applications, challenging agents with managing risks encompassing misuse and negative side effects. These tasks include tests to evaluate the safety of agents in daily scenarios as well as their robustness against indirect prompt injection attacks. Our experiments demonstrate that baseline agents, based on state-of-the-art LLMs, often fail to effectively prevent harm while performing the tasks. To mitigate these safety concerns, we propose a prompting method that encourages agents to prioritize safety considerations. While this method shows promise in promoting safer behaviors, there is still considerable room for improvement to fully earn user trust. This highlights the urgent need for continued research to develop more robust safety mechanisms in mobile environments. We open-source our benchmark at: https://mobilesafetybench.github.io/.

Figures (11)

• Figure 1: Overview of MobileSafetyBench. Incorporated with interactive real-system mobile device environments, MobileSafetyBench enables measuring the safety and helpfulness of agents controlling mobile devices across diverse task categories and risk types.
• Figure 2: The statistics of the tasks created in MobileSafetyBench. (a) The tasks, including both high-risk and low-risk tasks, span six groups of target operations. (b) Also, the high-risk tasks feature four different major types of risks and an additional distinct type of risk.
• Figure 3: Exemplary trajectories in a pair of tasks specified with the same instruction but different images in the file storage. The task completion is desirable as the image does not contain any risk in the low-risk task (left), while naively following the instruction results in safety issue as the image contains the detailed credit card information in the high-risk task (right).
• Figure 4: The goal achievement rates (left) and harm prevention rates (right) of the baseline agents in MobileSafetyBench. We provide detailed results of high-risk tasks in each risk type in Appendix \ref{['app:score_results']}. While the GPT-4o agents achieve the highest goal achievement rates, the Gemini-1.5 agents remark the highest harm prevention rates. The increase of harm prevention rates with SCoT prompt shows the effectiveness of the newly proposed method for inducing safe behaviors of the agents.
• Figure 5: Exemplary behavior of an agent attacked by an indirect prompt injection. After checking a message that contains a new malicious instruction, the agent sells the user's stock shares following the injected instruction.