Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Safe Robot Foundation Models Using Inductive Biases

Maximilian Tölle, Theo Gruner, Daniel Palenicek, Tim Schneider, Jonas Günster, Joe Watson, Davide Tateo, Puze Liu, Jan Peters

TL;DR

This work addresses the lack of formal safety guarantees in robot foundation models by introducing ATACOM, a safety layer that enforces geometric constraints after the base policy. By operating on the tangent space of a constraint manifold and integrating visual-derived constraints, the approach provides forward-invariant safety and input-to-state stability with minimal need for safety-specific fine-tuning. Empirical results in quasi-static manipulation and dynamic air-hockey tasks demonstrate that safety can be achieved without substantial loss in task performance, and the method scales to real-world deployment. The work highlights a practical pathway for safe deployment of generalist robotic policies, while acknowledging assumptions about known constraints and perception requirements and outlining avenues for automation and broader applicability.

Abstract

Safety is a critical requirement for the real-world deployment of robotic systems. Unfortunately, while current robot foundation models show promising generalization capabilities across a wide variety of tasks, they fail to address safety, an important aspect for ensuring long-term operation. Current robot foundation models assume that safe behavior should emerge by learning from a sufficiently large dataset of demonstrations. However, this approach has two clear major drawbacks. Firstly, there are no formal safety guarantees for a behavior cloning policy trained using supervised learning. Secondly, without explicit knowledge of any safety constraints, the policy may require an unreasonable number of additional demonstrations to even approximate the desired constrained behavior. To solve these key issues, we show how we can instead combine robot foundation models with geometric inductive biases using ATACOM, a safety layer placed after the foundation policy that ensures safe state transitions by enforcing action constraints. With this approach, we can ensure formal safety guarantees for generalist policies without providing extensive demonstrations of safe behavior, and without requiring any specific fine-tuning for safety. Our experiments show that our approach can be beneficial both for classical manipulation tasks, where we avoid unwanted collisions with irrelevant objects, and for dynamic tasks, such as the robot air hockey environment, where we can generate fast trajectories respecting complex tasks and joint space constraints.

Towards Safe Robot Foundation Models Using Inductive Biases

TL;DR

This work addresses the lack of formal safety guarantees in robot foundation models by introducing ATACOM, a safety layer that enforces geometric constraints after the base policy. By operating on the tangent space of a constraint manifold and integrating visual-derived constraints, the approach provides forward-invariant safety and input-to-state stability with minimal need for safety-specific fine-tuning. Empirical results in quasi-static manipulation and dynamic air-hockey tasks demonstrate that safety can be achieved without substantial loss in task performance, and the method scales to real-world deployment. The work highlights a practical pathway for safe deployment of generalist robotic policies, while acknowledging assumptions about known constraints and perception requirements and outlining avenues for automation and broader applicability.

Abstract

Safety is a critical requirement for the real-world deployment of robotic systems. Unfortunately, while current robot foundation models show promising generalization capabilities across a wide variety of tasks, they fail to address safety, an important aspect for ensuring long-term operation. Current robot foundation models assume that safe behavior should emerge by learning from a sufficiently large dataset of demonstrations. However, this approach has two clear major drawbacks. Firstly, there are no formal safety guarantees for a behavior cloning policy trained using supervised learning. Secondly, without explicit knowledge of any safety constraints, the policy may require an unreasonable number of additional demonstrations to even approximate the desired constrained behavior. To solve these key issues, we show how we can instead combine robot foundation models with geometric inductive biases using ATACOM, a safety layer placed after the foundation policy that ensures safe state transitions by enforcing action constraints. With this approach, we can ensure formal safety guarantees for generalist policies without providing extensive demonstrations of safe behavior, and without requiring any specific fine-tuning for safety. Our experiments show that our approach can be beneficial both for classical manipulation tasks, where we avoid unwanted collisions with irrelevant objects, and for dynamic tasks, such as the robot air hockey environment, where we can generate fast trajectories respecting complex tasks and joint space constraints.
Paper Structure (20 sections, 8 equations, 5 figures, 3 tables)

This paper contains 20 sections, 8 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. A Safety Module for Robot Foundation Models
  3. Acting on the Tangent Space of the Constraint Manifold
  4. Visual Constraint Generation
  5. Experiments
  6. Quasi-Static Pick-and-Place Environment
  7. Dynamic Air Hockey Environment
  8. Related Work
  9. Discussion
  10. Experimental Details: Pick-and-Place Tasks with Franka Robot
  11. Expert Data Collection & Fine-Tuning
  12. Pick-and-Place Franka Safety Layer
  13. Evaluation Task Descriptions
  14. pick-fruit-easy.
  15. pick-ball-medium.
  16. ...and 5 more sections

Figures (5)

  • Figure 1: Our proposed safety layer can be added to the output of an arbitrary rfm, e.g., $\pi_0$. (left) Without the added atacom safety layer, the vanilla $\pi_0$ policy crashes the robot into the table. We highlight the importance of the safety layer by plotting the impact of the $\pi_0$ action on the end effector's vertical velocity. While vanilla $\pi_0$ corrects the z position after the impact (red), the safety layer would have engaged earlier to circumvent crashing into the table. (right) Deployment with the safety layer results in a safe rollout without pronounced z-correction.
  • Figure 2: (left) Spheres cover the robot's hull at critical areas to formulate distance-based constraints ensuring safe executions of the vla action predictions. (middle) Bounding boxes of obstacles are generated from 2D instance segmentation and depth information. (right) We calculate the distance between the covering spheres and the obstacle's bounding box by projecting the sphere's center into the bounding box's coordinate frame and estimating the distance to the bounding box's hull.
  • Figure 3: Results in the manipulation tasks. Dashed histograms indicate the rfm combined with the atacom safety layers, while the solid ones represent the vanilla $\pi_0$ model. We report success rate, success rate for safe trajectories, percentage of safe trajectories among the successful ones, and normalized execution time. Results show that the safety layer does not impact heavily the success rate, while ensuring safety.
  • Figure 4: Video frame extracts from a rollout on three different tasks. Difficulty is dictated by the number of obstacles in the scene.
  • Figure 5: Safety violations of the octo policy w/o the safety module on the air hockey hitting task for different checkpoints during the training phase. We report the maximum constraint violation and the success rate of the robot hitting the puck into the goal over 500 episodes in simulation. When the atacom safety module is added, the policy remains compliant with safety constraints throughout fine-tuning, whereas the unmodified octo policy continues to breach safety limits. Both policies progressively improve their success rates over the number of fine-tuning steps.