SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systems

TL;DR

This work addresses safety challenges in LLM-integrated embodied AI for mobile robots by introducing SafeEmbodAI, a framework combining secure prompting, state management, and safety validation to mitigate malicious prompts and unsafe actions. It defines a Threat Model spanning Perception, Brain, and Action and proposes a three-pronged method to ensure safe reasoning over multi-modal data. A novel Mission Oriented Exploration Rate (MOER) metric and comprehensive experiments in EyeSim demonstrate substantial robustness gains under attack (e.g., up to 267% MOER improvement in complex environments), along with improved attack detection and reduced target loss, at a manageable computational cost. The results support SafeEmbodAI as a practical safety layer for deploying LLM-driven embodied AI in dynamic settings, with future work focusing on secure prompting strategies, broader prompt-attack analyses, and real-world physical validations.

Abstract

Embodied AI systems, including AI-powered robots that autonomously interact with the physical world, stand to be significantly advanced by Large Language Models (LLMs), which enable robots to better understand complex language commands and perform advanced tasks with enhanced comprehension and adaptability, highlighting their potential to improve embodied AI capabilities. However, this advancement also introduces safety challenges, particularly in robotic navigation tasks. Improper safety management can lead to failures in complex environments and make the system vulnerable to malicious command injections, resulting in unsafe behaviours such as detours or collisions. To address these issues, we propose \textit{SafeEmbodAI}, a safety framework for integrating mobile robots into embodied AI systems. \textit{SafeEmbodAI} incorporates secure prompting, state management, and safety validation mechanisms to secure and assist LLMs in reasoning through multi-modal data and validating responses. We designed a metric to evaluate mission-oriented exploration, and evaluations in simulated environments demonstrate that our framework effectively mitigates threats from malicious commands and improves performance in various environment settings, ensuring the safety of embodied AI systems. Notably, In complex environments with mixed obstacles, our method demonstrates a significant performance increase of 267\% compared to the baseline in attack scenarios, highlighting its robustness in challenging conditions.

Figures (8)

• Figure 1: A general Architecture of the Embodied AI System
• Figure 2: The Workflow of the Proposed Safety Framework
• Figure 3: Simulation Environments
• Figure 4: Example Outcomes of Experimental Trials
• Figure 5: Mission Oriented Exploration Rate Comparison