LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios

TL;DR

LPS-Bench introduces a first-of-its-kind benchmark for evaluating safety awareness in long-horizon, MCP-based computer-use agents by considering both benign and adversarial risk types across 65 scenarios in 7 task domains. It combines a scalable, human-in-the-loop data-generation pipeline with an automated LLM-based evaluation (LLM-as-a-judge) to assess planning-time safety along entire trajectories, not just end outcomes. Experiments with 13 diverse models reveal substantial planning-time safety gaps, especially under ambiguity and adversarial manipulation, and show that simple prompt-based mitigations offer only limited gains. The work provides an open-source framework, detailed risk taxonomy, and mitigation guidance to advance intrinsically safe and robust autonomous agents in real-world long-horizon planning tasks.

Abstract

Computer-use agents (CUAs) that interact with real computer systems can perform automated tasks but face critical safety risks. Ambiguous instructions may trigger harmful actions, and adversarial users can manipulate tool execution to achieve malicious goals. Existing benchmarks mostly focus on short-horizon or GUI-based tasks, evaluating on execution-time errors but overlooking the ability to anticipate planning-time risks. To fill this gap, we present LPS-Bench, a benchmark that evaluates the planning-time safety awareness of MCP-based CUAs under long-horizon tasks, covering both benign and adversarial interactions across 65 scenarios of 7 task domains and 9 risk types. We introduce a multi-agent automated pipeline for scalable data generation and adopt an LLM-as-a-judge evaluation protocol to assess safety awareness through the planning trajectory. Experiments reveal substantial deficiencies in existing CUAs' ability to maintain safe behavior. We further analyze the risks and propose mitigation strategies to improve long-horizon planning safety in MCP-based CUA systems. We open-source our code at https://github.com/tychenn/LPS-Bench.

Figures (27)

• Figure 1: The total safety scores of 13 tested LLM agents on LPS-Bench.
• Figure 2: Categories of agent safety benchmarks along two dimensions: interaction modality (GUI vs. MCP) and task horizon. LPS-Bench uniquely targets long-horizon safety challenges in MCP-based CUA.
• Figure 3: Overview of LPS-Bench. The framework illustrates the pipeline for multi-agent test case generation and the subsequent automated evaluation via LLM-as-a-judge. Additional test cases with complete trajectory examples are presented in Appendix \ref{['appendix:case_study']} for further illustration.
• Figure 4: Fine-grained distribution of test scenarios.
• Figure 5: The relevance of the models' instruction-following capability and planning safety awareness on LPS-Bench.