Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Geometric Fabrics: a Safe Guiding Medium for Policy Learning

Karl Van Wyk, Ankur Handa, Viktor Makoviychuk, Yijie Guo, Arthur Allshire, Nathan D. Ratliff

TL;DR

This work tackles the difficulty of learning dexterous manipulation under complex second-order robot dynamics by introducing geometric fabrics as a safe guiding medium that augments RL policies. It formalizes a fabric-based behavioral dynamics framework, linking artificial fabric dynamics to real robot dynamics via a torque controller and a policy-driven force $\boldsymbol{f}_\pi(\boldsymbol{a})$. The approach is instantiated for dexterous in-hand cube reorientation on a 16-actuator hand, with a detailed fabric design (attraction, repulsion, energization, damping) and an action-to-force mapping that enables acceleration and jerk constraint handling. Empirical results show state-of-the-art performance in simulation and strong sim-to-real transfer under disturbances, with reduced high-frequency action content compared to prior methods, demonstrating the practical impact of integrating learning with geometric-control-inspired dynamics.

Abstract

Robotics policies are always subjected to complex, second order dynamics that entangle their actions with resulting states. In reinforcement learning (RL) contexts, policies have the burden of deciphering these complicated interactions over massive amounts of experience and complex reward functions to learn how to accomplish tasks. Moreover, policies typically issue actions directly to controllers like Operational Space Control (OSC) or joint PD control, which induces straightline motion towards these action targets in task or joint space. However, straightline motion in these spaces for the most part do not capture the rich, nonlinear behavior our robots need to exhibit, shifting the burden of discovering these behaviors more completely to the agent. Unlike these simpler controllers, geometric fabrics capture a much richer and desirable set of behaviors via artificial, second order dynamics grounded in nonlinear geometry. These artificial dynamics shift the uncontrolled dynamics of a robot via an appropriate control law to form behavioral dynamics. Behavioral dynamics unlock a new action space and safe, guiding behavior over which RL policies are trained. Behavioral dynamics enable bang-bang-like RL policy actions that are still safe for real robots, simplify reward engineering, and help sequence real-world, high-performance policies. We describe the framework more generally and create a specific instantiation for the problem of dexterous, in-hand reorientation of a cube by a highly actuated robot hand.

Geometric Fabrics: a Safe Guiding Medium for Policy Learning

TL;DR

This work tackles the difficulty of learning dexterous manipulation under complex second-order robot dynamics by introducing geometric fabrics as a safe guiding medium that augments RL policies. It formalizes a fabric-based behavioral dynamics framework, linking artificial fabric dynamics to real robot dynamics via a torque controller and a policy-driven force . The approach is instantiated for dexterous in-hand cube reorientation on a 16-actuator hand, with a detailed fabric design (attraction, repulsion, energization, damping) and an action-to-force mapping that enables acceleration and jerk constraint handling. Empirical results show state-of-the-art performance in simulation and strong sim-to-real transfer under disturbances, with reduced high-frequency action content compared to prior methods, demonstrating the practical impact of integrating learning with geometric-control-inspired dynamics.

Abstract

Robotics policies are always subjected to complex, second order dynamics that entangle their actions with resulting states. In reinforcement learning (RL) contexts, policies have the burden of deciphering these complicated interactions over massive amounts of experience and complex reward functions to learn how to accomplish tasks. Moreover, policies typically issue actions directly to controllers like Operational Space Control (OSC) or joint PD control, which induces straightline motion towards these action targets in task or joint space. However, straightline motion in these spaces for the most part do not capture the rich, nonlinear behavior our robots need to exhibit, shifting the burden of discovering these behaviors more completely to the agent. Unlike these simpler controllers, geometric fabrics capture a much richer and desirable set of behaviors via artificial, second order dynamics grounded in nonlinear geometry. These artificial dynamics shift the uncontrolled dynamics of a robot via an appropriate control law to form behavioral dynamics. Behavioral dynamics unlock a new action space and safe, guiding behavior over which RL policies are trained. Behavioral dynamics enable bang-bang-like RL policy actions that are still safe for real robots, simplify reward engineering, and help sequence real-world, high-performance policies. We describe the framework more generally and create a specific instantiation for the problem of dexterous, in-hand reorientation of a cube by a highly actuated robot hand.
Paper Structure (20 sections, 12 equations, 1 figure, 2 tables)

This paper contains 20 sections, 12 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Formulation
  3. Forcing Energized Fabrics
  4. Behavioral Dynamics
  5. Torque Controller
  6. Application to Multi-Fingered Cube Reorientation
  7. Fabrics Design and Policy Action Space
  8. Attraction
  9. Repulsion
  10. Energization
  11. Geometrically-Consistent Damping
  12. Acceleration and Jerk Handling
  13. Action Space
  14. Numerical Integration
  15. Reinforcement Learning Setup
  16. ...and 5 more sections

Figures (1)

  • Figure 2: Spectral content of target joint angles generated by the (a) FGP and (b) DeXtreme policies indicating greater noise attenuation in FGP control. Blue curve is mean amplitude, red curves are the minimum and maximum amplitudes.