Robots that Suggest Safe Alternatives

TL;DR

The paper addresses the risk that goal-conditioned robot policies can fail safely or perform poorly on out-of-distribution goals. It introduces SALT, a framework that performs offline, goal-conditioned reach-avoid analysis using Hamilton-Jacobi reachability to produce a reach-avoid value function $V_*^\pi(s; g)$, which serves as a safety filter online over the user’s goal. When a given goal is unsafe or non-lively, SALT optimizes over alternative goals in the same space and presents safe options to the human, ensuring comparable task progress. The approach is demonstrated on indoor navigation and Franka Panda tabletop manipulation with both discrete and continuous goal representations, showing improved failure detection and human-aligned alternatives, thereby enabling safer, more proactive human-robot interaction.

Abstract

Goal-conditioned policies, such as those learned via imitation learning, provide an easy way for humans to influence what tasks robots accomplish. However, these robot policies are not guaranteed to execute safely or to succeed when faced with out-of-distribution requests. In this work, we enable robots to know when they can confidently execute a user's desired goal, and automatically suggest safe alternatives when they cannot. Our approach is inspired by control-theoretic safety filtering, wherein a safety filter minimally adjusts a robot's candidate action to be safe. Our key idea is to pose alternative suggestion as a safe control problem in goal space, rather than in action space. Offline, we use reachability analysis to compute a goal-parameterized reach-avoid value network which quantifies the safety and liveness of the robot's pre-trained policy. Online, our robot uses the reach-avoid value network as a safety filter, monitoring the human's given goal and actively suggesting alternatives that are similar but meet the safety specification. We demonstrate our Safe ALTernatives (SALT) framework in simulation experiments with indoor navigation and Franka Panda tabletop manipulation, and with both discrete and continuous goal representations. We find that SALT is able to learn to predict successful and failed closed-loop executions, is a less pessimistic monitor than open-loop uncertainty quantification, and proposes alternatives that consistently align with those people find acceptable.

Figures (2)

• Figure 1: Figure 1: Safe ALTernatives (SALT). If a robot naively executes a user's request, it can fail for a variety of reasons. For example, a request to pick up the red bowl leads the robot to fail to grasp. Our SALT framework enables a robot to detect if it can successfully accomplish a user's original goal; if it cannot, the robot automatically proposes an alternative it can safely succeed at (e.g., get brown bowl). Videos on the project website: https://cmu-intentlab.github.io/salt/.
• Figure 2: Figure 2: Robots that Suggest Safe Alternatives (SALT) Framework. (Left) Offline, a reach-avoid value network is learned to estimate the safety and liveness properties of a pre-trained goal-conditioned robot policy. (Right) Online, a human inputs a desired goal, which is first monitored by our reach-avoid value function. If the input goal satisfies both safety and liveness, then the policy is executed. Otherwise, the robot solves a safe control problem over alternative goals (e.g., objects in the scene) to propose an alternative. If the human accepts, then the robot confidently executes on the new goal.