Hazards in Daily Life? Enabling Robots to Proactively Detect and Resolve Anomalies

TL;DR

This work introduces a multi-agent brainstorming approach, where agents collaborate and generate diverse scenarios covering household hazards, hygiene management, and child safety, and demonstrates that the generated environment outperforms others in terms of task description and scene diversity, ultimately enabling robotic agents to better address potential household hazards.

Abstract

Existing household robots have made significant progress in performing routine tasks, such as cleaning floors or delivering objects. However, a key limitation of these robots is their inability to recognize potential problems or dangers in home environments. For example, a child may pick up and ingest medication that has fallen on the floor, posing a serious risk. We argue that household robots should proactively detect such hazards or anomalies within the home, and propose the task of anomaly scenario generation. We leverage foundational models instead of relying on manually labeled data to build simulated environments. Specifically, we introduce a multi-agent brainstorming approach, where agents collaborate and generate diverse scenarios covering household hazards, hygiene management, and child safety. These textual task descriptions are then integrated with designed 3D assets to simulate realistic environments. Within these constructed environments, the robotic agent learns the necessary skills to proactively discover and handle the proposed anomalies through task decomposition, and optimal learning approach selection. We demonstrate that our generated environment outperforms others in terms of task description and scene diversity, ultimately enabling robotic agents to better address potential household hazards.

Figures (6)

• Figure 1: Comparison of passively instructed robots and our proactive detection robot. Our paradigm creates benefits and convenience for safety, even in the absence of human presence.
• Figure 2: AnomalyGen includes 3 modules: a) Group Brainstorming, b) Anomalous Scenarios Generation, c) Proactive Anomaly Detection and Anomaly Task Learning.
• Figure 3: Snapshots of the learned skills across 4 exemplary long-horizon sequential tasks and 1 single-step task.
• Figure 4: Distribution of types of anomalous scenarios. The red color represents "Household Hazards," the blue color denotes "Hygiene Management," and the green color denotes "Child Safety Measures."
• Figure 5: Anomaly resolution completion rate across different categories.