Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task

Xiaoya Lu, Yijin Zhou, Zeren Chen, Ruocheng Wang, Bingrui Sima, Enshen Zhou, Lu Sheng, Dongrui Liu, Jing Shao

Abstract

Vision-Language Models (VLMs) empower embodied agents to execute complex instructions, yet they remain vulnerable to contextual safety risks where benign commands become hazardous due to subtle environmental states. Existing safeguards often prove inadequate. Rule-based methods lack scalability in object-dense scenes, whereas model-based approaches relying on prompt engineering suffer from unfocused perception, resulting in missed risks or hallucinations. To address this, we propose an architecture-agnostic safeguard featuring Context-Guided Chain-of-Thought (CG-CoT). This mechanism decomposes risk assessment into active perception that sequentially anchors attention to interaction targets and relevant spatial neighborhoods, followed by semantic judgment based on this visual evidence. We support this approach with a curated grounding dataset and a two-stage training strategy utilizing Reinforcement Fine-Tuning (RFT) with process rewards to enforce precise intermediate grounding. Experiments demonstrate that our model HomeGuard significantly enhances safety, improving risk match rates by over 30% compared to base models while reducing oversafety. Beyond hazard detection, the generated visual anchors serve as actionable spatial constraints for downstream planners, facilitating explicit collision avoidance and safety trajectory generation. Code and data are released under https://github.com/AI45Lab/HomeGuard

HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task

Abstract

Vision-Language Models (VLMs) empower embodied agents to execute complex instructions, yet they remain vulnerable to contextual safety risks where benign commands become hazardous due to subtle environmental states. Existing safeguards often prove inadequate. Rule-based methods lack scalability in object-dense scenes, whereas model-based approaches relying on prompt engineering suffer from unfocused perception, resulting in missed risks or hallucinations. To address this, we propose an architecture-agnostic safeguard featuring Context-Guided Chain-of-Thought (CG-CoT). This mechanism decomposes risk assessment into active perception that sequentially anchors attention to interaction targets and relevant spatial neighborhoods, followed by semantic judgment based on this visual evidence. We support this approach with a curated grounding dataset and a two-stage training strategy utilizing Reinforcement Fine-Tuning (RFT) with process rewards to enforce precise intermediate grounding. Experiments demonstrate that our model HomeGuard significantly enhances safety, improving risk match rates by over 30% compared to base models while reducing oversafety. Beyond hazard detection, the generated visual anchors serve as actionable spatial constraints for downstream planners, facilitating explicit collision avoidance and safety trajectory generation. Code and data are released under https://github.com/AI45Lab/HomeGuard
Paper Structure (36 sections, 3 equations, 10 figures, 4 tables)

This paper contains 36 sections, 3 equations, 10 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. VLM-driven Embodied Agents
  4. Safety Evaluation and Alignment of Embodied Agents
  5. Safety Guardrails for Embodied Agents
  6. Method
  7. Context-Guidied Chain-of-Thought
  8. Training Strategy
  9. Dataset
  10. Experiment
  11. Setup
  12. Performance in
  13. Performance in Public Risk Identification Benchmark
  14. Ablation Study
  15. Application: Safety-Aware Planning and Trajectory Generation
  16. ...and 21 more sections

Figures (10)

  • Figure 1: Identifying implicit contextual risks via Context-Guided Chain-of-Thought.
  • Figure 2: The two-stage training pipeline and a visualization of the sequential reasoning process for detecting risk identification in household tasks.
  • Figure 3: (a) The four-stage data collection pipeline for generating the dataset. (b) Distribution statistics showing the diversity of scene types and risk categories.
  • Figure 4: Comparison of inference efficiency and safety performance, where the x-axis is plotted on a logarithmic scale.
  • Figure 5: An application case of facilitating safe trajectory generation. The underlying motion trajectories are generated by RoboBrain2.5-8B tan2026robobrain.
  • ...and 5 more figures