Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Can AI Perceive Physical Danger and Intervene?

Abhishek Jindal, Dmitry Kalashnikov, R. Alex Hofer, Oscar Chang, Divya Garikapati, Anirudha Majumdar, Pierre Sermanet, Vikas Sindhwani

TL;DR

The paper addresses physical safety in embodied AI by introducing ASIMOV-2.0, a multimodal benchmark grounded in real injury narratives and industrial safety standards. It employs a Generator-Critic-Refine data pipeline to synthesize Injury text, Constraint images, and Safety videos across text, image, and video modalities. Across large frontier models, it reveals modality and embodiment gaps, high constraint-violation rates, and latency-accuracy tradeoffs, and demonstrates that thinking-based post-training yields state-of-the-art constraint satisfaction with more concise, interpretable thinking traces. The work advances safe deployment of embodied AI and provides a pathway toward standards-aligned development and evaluation.

Abstract

When AI interacts with the physical world -- as a robot or an assistive agent -- new safety challenges emerge beyond those of purely ``digital AI". In such interactions, the potential for physical harm is direct and immediate. How well do state-of-the-art foundation models understand common-sense facts about physical safety, e.g. that a box may be too heavy to lift, or that a hot cup of coffee should not be handed to a child? In this paper, our contributions are three-fold: first, we develop a highly scalable approach to continuous physical safety benchmarking of Embodied AI systems, grounded in real-world injury narratives and operational safety constraints. To probe multi-modal safety understanding, we turn these narratives and constraints into photorealistic images and videos capturing transitions from safe to unsafe states, using advanced generative models. Secondly, we comprehensively analyze the ability of major foundation models to perceive risks, reason about safety, and trigger interventions; this yields multi-faceted insights into their deployment readiness for safety-critical agentic applications. Finally, we develop a post-training paradigm to teach models to explicitly reason about embodiment-specific safety constraints provided through system instructions. The resulting models generate thinking traces that make safety reasoning interpretable and transparent, achieving state of the art performance in constraint satisfaction evaluations. The benchmark is released at https://asimov-benchmark.github.io/v2

Can AI Perceive Physical Danger and Intervene?

TL;DR

The paper addresses physical safety in embodied AI by introducing ASIMOV-2.0, a multimodal benchmark grounded in real injury narratives and industrial safety standards. It employs a Generator-Critic-Refine data pipeline to synthesize Injury text, Constraint images, and Safety videos across text, image, and video modalities. Across large frontier models, it reveals modality and embodiment gaps, high constraint-violation rates, and latency-accuracy tradeoffs, and demonstrates that thinking-based post-training yields state-of-the-art constraint satisfaction with more concise, interpretable thinking traces. The work advances safe deployment of embodied AI and provides a pathway toward standards-aligned development and evaluation.

Abstract

When AI interacts with the physical world -- as a robot or an assistive agent -- new safety challenges emerge beyond those of purely ``digital AI". In such interactions, the potential for physical harm is direct and immediate. How well do state-of-the-art foundation models understand common-sense facts about physical safety, e.g. that a box may be too heavy to lift, or that a hot cup of coffee should not be handed to a child? In this paper, our contributions are three-fold: first, we develop a highly scalable approach to continuous physical safety benchmarking of Embodied AI systems, grounded in real-world injury narratives and operational safety constraints. To probe multi-modal safety understanding, we turn these narratives and constraints into photorealistic images and videos capturing transitions from safe to unsafe states, using advanced generative models. Secondly, we comprehensively analyze the ability of major foundation models to perceive risks, reason about safety, and trigger interventions; this yields multi-faceted insights into their deployment readiness for safety-critical agentic applications. Finally, we develop a post-training paradigm to teach models to explicitly reason about embodiment-specific safety constraints provided through system instructions. The resulting models generate thinking traces that make safety reasoning interpretable and transparent, achieving state of the art performance in constraint satisfaction evaluations. The benchmark is released at https://asimov-benchmark.github.io/v2

Paper Structure

This paper contains 10 sections, 12 figures, 6 tables.

Table of Contents

  1. Introduction
  2. ASIMOV-2.0: A New Physical Safety Benchmark
  3. Evaluations and Insights
  4. "Thinking" for Safety
  5. Related Work
  6. Conclusion
  7. Examples of Thinking Traces before and after finetuning
  8. Proactive vs Reactive Interventions
  9. NEISS injury types and physical constraint taxonomy
  10. Eval Datasets - descriptive statistics

Figures (12)

  • Figure 1: ASIMOV-2.0 Physical Safety Benchmark Components and Key Questions
  • Figure 2: Pipeline for generating Asimov-2.0 scenarios and labels. All scenarios are grounded in real-world injury reports and a taxonomy of operational safety constraints.
  • Figure 4: ASIMOV-2.0-Injury: Evaluation Results
  • Figure 5: ASIMOV-2.0-Video: Evaluation Results
  • Figure 6: ASIMOV-Constraints: Results
  • ...and 7 more figures