Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FLEX: A Framework for Learning Robot-Agnostic Force-based Skills Involving Sustained Contact Object Manipulation

Shijie Fang, Wenchang Gao, Shivam Goel, Christopher Thierauf, Matthias Scheutz, Jivko Sinapov

TL;DR

FLEX tackles sustained contact manipulation across diverse objects and robots by learning force-based, object-centric policies that operate in the object's force space rather than robot kinematics. It decouples policy learning from robot dynamics, trains two TD3-based policies for prismatic and revolute joints in simulation on representative articulated objects, and enables direct transfer to different robots without retraining. Joint-type inference during execution uses MLE on end-effector trajectories with PCA initialization, supporting robot-agnostic deployment. Empirical results in Robosuite show superior training efficiency, robust generalization to unseen objects, and successful cross-robot and real-world transfer, including a UR5 drawer demonstration.

Abstract

Learning to manipulate objects efficiently, particularly those involving sustained contact (e.g., pushing, sliding) and articulated parts (e.g., drawers, doors), presents significant challenges. Traditional methods, such as robot-centric reinforcement learning (RL), imitation learning, and hybrid techniques, require massive training and often struggle to generalize across different objects and robot platforms. We propose a novel framework for learning object-centric manipulation policies in force space, decoupling the robot from the object. By directly applying forces to selected regions of the object, our method simplifies the action space, reduces unnecessary exploration, and decreases simulation overhead. This approach, trained in simulation on a small set of representative objects, captures object dynamics -- such as joint configurations -- allowing policies to generalize effectively to new, unseen objects. Decoupling these policies from robot-specific dynamics enables direct transfer to different robotic platforms (e.g., Kinova, Panda, UR5) without retraining. Our evaluations demonstrate that the method significantly outperforms baselines, achieving over an order of magnitude improvement in training efficiency compared to other state-of-the-art methods. Additionally, operating in force space enhances policy transferability across diverse robot platforms and object types. We further showcase the applicability of our method in a real-world robotic setting. For supplementary materials and videos, please visit: https://tufts-ai-robotics-group.github.io/FLEX/

FLEX: A Framework for Learning Robot-Agnostic Force-based Skills Involving Sustained Contact Object Manipulation

TL;DR

FLEX tackles sustained contact manipulation across diverse objects and robots by learning force-based, object-centric policies that operate in the object's force space rather than robot kinematics. It decouples policy learning from robot dynamics, trains two TD3-based policies for prismatic and revolute joints in simulation on representative articulated objects, and enables direct transfer to different robots without retraining. Joint-type inference during execution uses MLE on end-effector trajectories with PCA initialization, supporting robot-agnostic deployment. Empirical results in Robosuite show superior training efficiency, robust generalization to unseen objects, and successful cross-robot and real-world transfer, including a UR5 drawer demonstration.

Abstract

Learning to manipulate objects efficiently, particularly those involving sustained contact (e.g., pushing, sliding) and articulated parts (e.g., drawers, doors), presents significant challenges. Traditional methods, such as robot-centric reinforcement learning (RL), imitation learning, and hybrid techniques, require massive training and often struggle to generalize across different objects and robot platforms. We propose a novel framework for learning object-centric manipulation policies in force space, decoupling the robot from the object. By directly applying forces to selected regions of the object, our method simplifies the action space, reduces unnecessary exploration, and decreases simulation overhead. This approach, trained in simulation on a small set of representative objects, captures object dynamics -- such as joint configurations -- allowing policies to generalize effectively to new, unseen objects. Decoupling these policies from robot-specific dynamics enables direct transfer to different robotic platforms (e.g., Kinova, Panda, UR5) without retraining. Our evaluations demonstrate that the method significantly outperforms baselines, achieving over an order of magnitude improvement in training efficiency compared to other state-of-the-art methods. Additionally, operating in force space enhances policy transferability across diverse robot platforms and object types. We further showcase the applicability of our method in a real-world robotic setting. For supplementary materials and videos, please visit: https://tufts-ai-robotics-group.github.io/FLEX/

Paper Structure

This paper contains 36 sections, 17 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Articulated Object Parametrization
  5. Reinforcement Learning for Force-Based Manipulation
  6. Problem Formulation
  7. Learning Methodology
  8. Force-based skill learning
  9. Interactive Joint Parameter Estimation
  10. Robot Execution
  11. Experiments
  12. Experimental Setup
  13. Evaluation and Metrics
  14. Baselines
  15. Results & Discussion
  16. ...and 21 more sections

Figures (3)

  • Figure 1: Our method (FLEX: Force-based Learning for EXtended Manipulation) learns force-based skills for manipulating articulated objects across various robots by decoupling policy learning from robot dynamics. The blue box (top-left) shows the core mechanism of force-based skill learning, while the green box (top-right) illustrates the transfer to different robots. The white box (bottom) highlights manipulation tasks: L-R, the UR5e opens a cabinet, Kinova opens a microwave door, Panda opens a dishwasher rack, and a real UR5 opens a drawer. Detailed architecture in Figure \ref{['fig:overall_diagram']}.
  • Figure 2: Illustration of the two joint configurations and states
  • Figure 3: The figure presents the architecture of FLEX (Force-based Learning for EXtended Manipulation) for sustained contact manipulation of articulated objects. The blue boxes represent Force-based skill learning in simulation, the red boxes depict Interactive joint parameter estimation, and the green boxes show the Robot execution phase, where the learned policies are applied to perform tasks.