Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Safety Challenge of World Models for Embodied AI Agents: A Review

Lorenzo Baraldi, Zifan Zeng, Chongzhe Zhang, Aradhana Nayak, Hongbo Zhu, Feng Liu, Qunli Zhang, Peng Wang, Shiming Liu, Zheng Hu, Angelo Cangelosi, Lorenzo Baraldi

TL;DR

This review addresses the safety challenges of World Models in embodied AI by surveying scene-generation and control WM approaches in autonomous driving and robotics, and by proposing a pathology-based safety taxonomy. It combines a structured literature analysis with empirical predictions from SoTA models to identify common fault modes and to quantify safety-related pathologies using multimodal and task-specific metrics. The key contributions include a formal pathology criteria framework, a structured experimental protocol, and a forward-looking set of directions (metrics, neurosymbolic control, and MLLM feedback) to improve safety and reliability. The work highlights the gap between current WM predictions and real-world safety requirements, underscoring practical implications for deploying safe, robust embodied agents.

Abstract

The rapid progress in embodied artificial intelligence has highlighted the necessity for more advanced and integrated models that can perceive, interpret, and predict environmental dynamics. In this context, World Models (WMs) have been introduced to provide embodied agents with the abilities to anticipate future environmental states and fill in knowledge gaps, thereby enhancing agents' ability to plan and execute actions. However, when dealing with embodied agents it is fundamental to ensure that predictions are safe for both the agent and the environment. In this article, we conduct a comprehensive literature review of World Models in the domains of autonomous driving and robotics, with a specific focus on the safety implications of scene and control generation tasks. Our review is complemented by an empirical analysis, wherein we collect and examine predictions from state-of-the-art models, identify and categorize common faults (herein referred to as pathologies), and provide a quantitative evaluation of the results.

The Safety Challenge of World Models for Embodied AI Agents: A Review

TL;DR

This review addresses the safety challenges of World Models in embodied AI by surveying scene-generation and control WM approaches in autonomous driving and robotics, and by proposing a pathology-based safety taxonomy. It combines a structured literature analysis with empirical predictions from SoTA models to identify common fault modes and to quantify safety-related pathologies using multimodal and task-specific metrics. The key contributions include a formal pathology criteria framework, a structured experimental protocol, and a forward-looking set of directions (metrics, neurosymbolic control, and MLLM feedback) to improve safety and reliability. The work highlights the gap between current WM predictions and real-world safety requirements, underscoring practical implications for deploying safe, robust embodied agents.

Abstract

The rapid progress in embodied artificial intelligence has highlighted the necessity for more advanced and integrated models that can perceive, interpret, and predict environmental dynamics. In this context, World Models (WMs) have been introduced to provide embodied agents with the abilities to anticipate future environmental states and fill in knowledge gaps, thereby enhancing agents' ability to plan and execute actions. However, when dealing with embodied agents it is fundamental to ensure that predictions are safe for both the agent and the environment. In this article, we conduct a comprehensive literature review of World Models in the domains of autonomous driving and robotics, with a specific focus on the safety implications of scene and control generation tasks. Our review is complemented by an empirical analysis, wherein we collect and examine predictions from state-of-the-art models, identify and categorize common faults (herein referred to as pathologies), and provide a quantitative evaluation of the results.

Paper Structure

This paper contains 11 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Preliminaries on World Models
  3. World Models for Scene Generation
  4. World Models for Control
  5. Pathologies in World Models Safety
  6. Experimental Settings
  7. Pathology Criteria in WM for Scene Generation
  8. Pathology Criteria in WM for Control
  9. Metrics for World Models Safety
  10. Quantitative Results
  11. Conclusions and Future Directions

Figures (3)

  • Figure 1: Illustration of a World Model in the domain of autonomous driving or robotics. The current observation and conditions are used to predict future observations, on the tasks of novel scenes (yellow) or control actions (blue) generation. Our pathology criteria enable the evaluation of safety in the generated outputs for both tasks.
  • Figure 2: Illustration of pathologies identified in SoTA WMs for scene generation. Visual Quality (MagicDriveDit), Temporal Consistency (Open-Sora), Traffic Adherence (Comsos), Physical Conformity (Vista), Condition Consistency (This&That)
  • Figure 3: Illustration of pathologies identified in SoTA WMs for control: RoboGen (a), Octo (b), MILE (c), (d)