Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

See Something, Say Something: Context-Criticality-Aware Mobile Robot Communication for Hazard Mitigations

Bhavya Oza, Devam Shah, Ghanashyama Prabhu, Devika Kodi, Aliasghar Arab

Abstract

The proverb ``see something, say something'' captures a core responsibility of autonomous mobile robots in safety-critical situations: when they detect a hazard, they must communicate--and do so quickly. In emergency scenarios, delayed or miscalibrated responses directly increase the time to action and the risk of damage. We argue that a systematic context-sensitive assessment of the criticality level, time sensitivity, and feasibility of mitigation is necessary for AMRs to reduce time to action and respond effectively. This paper presents a framework in which VLM/LLM-based perception drives adaptive message generation, for example, a knife in a kitchen produces a calm acknowledgment; the same object in a corridor triggers an urgent coordinated alert. Validation in 60+ runs using a patrolling mobile robot not only empowers faster response, but also brings user trusts to 82\% compared to fixed-priority baselines, validating that structured criticality assessment improves both response speed and mitigation effectiveness.

See Something, Say Something: Context-Criticality-Aware Mobile Robot Communication for Hazard Mitigations

Abstract

The proverb ``see something, say something'' captures a core responsibility of autonomous mobile robots in safety-critical situations: when they detect a hazard, they must communicate--and do so quickly. In emergency scenarios, delayed or miscalibrated responses directly increase the time to action and the risk of damage. We argue that a systematic context-sensitive assessment of the criticality level, time sensitivity, and feasibility of mitigation is necessary for AMRs to reduce time to action and respond effectively. This paper presents a framework in which VLM/LLM-based perception drives adaptive message generation, for example, a knife in a kitchen produces a calm acknowledgment; the same object in a corridor triggers an urgent coordinated alert. Validation in 60+ runs using a patrolling mobile robot not only empowers faster response, but also brings user trusts to 82\% compared to fixed-priority baselines, validating that structured criticality assessment improves both response speed and mitigation effectiveness.

Paper Structure

This paper contains 19 sections, 15 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. PROBLEM FORMULATION
  3. Step 1 — Hazard Detection
  4. Step 2 — Contextual Factors
  5. Step 3 — Risk Assessment
  6. Step 4 — Criticality-Aware Communication
  7. Objective
  8. METHODOLOGY
  9. Hazard Detection Model
  10. Criticality-Driven Message Generation
  11. Latency and Time-to-Communication
  12. System Implementation and Hardware Architecture
  13. Multi-Party Coordination
  14. EXPERIMENTS AND RESULTS
  15. Scenarios: Context Determines What Gets Said
  16. ...and 4 more sections

Figures (4)

  • Figure 3: A patrolling AMR detecting a fallen person, correlating a nearby knife with heightened risk, and triggering a context-aware mitigation response.
  • Figure 4: Workflow of hazard detection, communication strategy generation, and multi-party coordination.
  • Figure 5: User Evaluation Breakdown (Trust, Understanding, Preference) for different scenarios
  • Figure 6: Scenario-wise risk scores $\rho_t$ (Eq. (\ref{['eq:criticality_threshold']})): Low $[0\text{--}4]$, Medium $[5\text{--}7]$, High $[8\text{--}10]$. Alarm activation occurs only for $\rho_t \geq 5$.

Theorems & Definitions (8)

  • Definition 1: Hazard Detection
  • Remark 1: Detection is Not Enough
  • Definition 2: Contextual Factors
  • Definition 3: Risk of Loss Criticality
  • Remark 2: Context-Dependence of Criticality
  • Definition 4: Communication Tuple
  • Remark 3: Context Disambiguation
  • Remark 4: LLM Consistency and Opacity