AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions
Zonghao Ying, Le Wang, Yisong Xiao, Jiakai Wang, Yuqing Ma, Jinyang Guo, Zhenfei Yin, Mingchuan Zhang, Aishan Liu, Xianglong Liu
TL;DR
AGENTSAFE introduces a holistic safety benchmark for embodied vision-language agents under hazardous instructions, uniting an adversarial sandbox (SAFE-THOR), a risk-aware task suite (SAFE-VERSE), and a multi-level evaluation protocol (SAFE-DIAGNOSE). It demonstrates systematic safety gaps, particularly in planning, across nine state-of-the-art VLMs and two workflows, and shows that proactive thought-level auditing (SAFE-AUDIT) can mitigate unsafe behavior at the source. The benchmark emphasizes end-to-end evaluation across perception, planning, and execution, enabling fine-grained diagnostics and safer embodied intelligence. Overall, AGENTSAFE provides a foundation for developing and evaluating safety mechanisms that generalize to real-world embodied AI systems.
Abstract
The integration of vision-language models (VLMs) is driving a new generation of embodied agents capable of operating in human-centered environments. However, as deployment expands, these systems face growing safety risks, particularly when executing hazardous instructions. Current safety evaluation benchmarks remain limited: they cover only narrow scopes of hazards and focus primarily on final outcomes, neglecting the agent's full perception-planning-execution process and thereby obscuring critical failure modes. Therefore, we present SAFE, a benchmark for systematically assessing the safety of embodied VLM agents on hazardous instructions. SAFE comprises three components: SAFE-THOR, an extensible adversarial simulation sandbox with a universal adapter that maps high-level VLM outputs to low-level embodied controls, supporting diverse agent workflow integration; SAFE-VERSE, a risk-aware task suite inspired by Asimov's Three Laws of Robotics, comprising 45 adversarial scenarios, 1,350 hazardous tasks, and 9,900 instructions that span risks to humans, environments, and agents; and SAFE-DIAGNOSE, a multi-level and fine-grained evaluation protocol measuring agent performance across perception, planning, and execution. Applying SAFE to nine state-of-the-art VLMs and two embodied agent workflows, we uncover systematic failures in translating hazard recognition into safe planning and execution. Our findings reveal fundamental limitations in current safety alignment and demonstrate the necessity of a comprehensive, multi-stage evaluation for developing safer embodied intelligence.