Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Making

TL;DR

This work introduces SAFEL, a modular evaluation framework for the physical safety of LLM-driven embodied decision making, paired with EmbodyGuard, a PDDL-grounded benchmark that tests both explicit malicious commands and context-dependent hazards. SAFEL decomposes planning safety into a Command Refusal Test and a three-stage Plan Safety Test (Goal Interpretation, Transition Modeling, and Action Sequencing), enabling fine-grained diagnosis of where safety failures occur in the planning pipeline. EmbodyGuard data are generated synthetically and validated through symbolic and human checks within the iGibson-based domain, allowing safe, executable plans to be tested by a diverse set of 13 state-of-the-art LLMs. Results show that while models adeptly reject overtly unsafe commands, they struggle with subtle risk reasoning, predicting environmental transitions, and arranging safe action sequences, highlighting substantial gaps and providing a foundation for targeted, modular improvements in safe embodied reasoning. The framework and benchmark thus offer a rigorous, interpretable path toward more reliable safety-aware decision making in real-world embodied AI systems, with clear avenues for future enhancements including RL-based learning and broader simulator integration.

Abstract

Large Language Models (LLMs) are increasingly used for decision making in embodied agents, yet existing safety evaluations often rely on coarse success rates and domain-specific setups, making it difficult to diagnose why and where these models fail. This obscures our understanding of embodied safety and limits the selective deployment of LLMs in high-risk physical environments. We introduce SAFEL, the framework for systematically evaluating the physical safety of LLMs in embodied decision making. SAFEL assesses two key competencies: (1) rejecting unsafe commands via the Command Refusal Test, and (2) generating safe and executable plans via the Plan Safety Test. Critically, the latter is decomposed into functional modules, goal interpretation, transition modeling, action sequencing, enabling fine-grained diagnosis of safety failures. To support this framework, we introduce EMBODYGUARD, a PDDL-grounded benchmark containing 942 LLM-generated scenarios covering both overtly malicious and contextually hazardous instructions. Evaluation across 13 state-of-the-art LLMs reveals that while models often reject clearly unsafe commands, they struggle to anticipate and mitigate subtle, situational risks. Our results highlight critical limitations in current LLMs and provide a foundation for more targeted, modular improvements in safe embodied reasoning.

Figures (5)

• Figure 1: Our SAFEL pipeline assesses physical safety across multiple stages using EMBODYGUARD scenarios, which consist of both malicious commands and contextually risky instructions. This example illustrates two failure modes: overlooking the hazardous effect of an action under unsafe preconditions, and executing otherwise safe actions in an unsafe order. All modules are evaluated independently, allowing us to isolate each stage of failure. As a result, our framework offers actionable insights into where and how embodied LLMs break down, enabling more targeted interventions for safety-critical applications.
• Figure 2: Plan Safety Test performance of LLMs. across planning stages using the SAFEL. Each radar plot evaluates Goal Interpretation, Transition Modeling, and Action Sequencing across two benchmark types: situational and malicious. Models exhibit significant weaknesses in Transition Modeling for both malicious and situational scenarios and in Action Sequencing for subtle and situational risk-related tasks. The exact meaning of each symbol is explained in the Section 3.
• Figure 3: Overview of the input-output structure of the SAFEL for evaluating embodied agents' safety awareness using the benchmark EMBODYGUARD, comprising two scenario categories --- Malicious and Situational. Each evaluation component within SAFEL (goal interpretation, safe precondition modeling, risky effect modeling, action sequencing, and command refusal) is illustrated with distinct inputs and outputs. SAFEL assesses agents' abilities to appropriately refuse unsafe commands, accurately interpret task goals, identify safe preconditions, model potentially unsafe effects, and correctly sequence actions.
• Figure 4: The PDDL verification and correction process. (1) Verifier checks for missing predicates and parameters, validates using the Fast Downward planner, and rejects erroneous PDDL. (2) Corrector applies rule-based fixes for recoverable rejected PDDL. (3) Workflow: (i) Initial verification produces either Filtered PDDL or Rejected PDDL, which is sent to the corrector. (ii) Corrected PDDL undergoes re-verification, resulting in either Corrected Filtered PDDL or Buried PDDL (if unfixable). (iii) Final PDDL is formed by merging Filtered PDDL and Corrected PDDL. If GT_Plan lacks safe, risk actions or has fewer than 3 steps, it is rejected. (iv) The final PDDL undergoes human review, resulting in the Human-Filtered Final PDDL.
• Figure :