Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DexForce: Extracting Force-informed Actions from Kinesthetic Demonstrations for Dexterous Manipulation

Claire Chen, Zhongchun Yu, Hojung Choi, Mark Cutkosky, Jeannette Bohg

TL;DR

DexForce introduces force-informed actions derived from kinesthetic demonstrations by augmenting observed fingertip positions with measured contact forces, enabling effective imitation learning for contact-rich dexterous manipulation. The method leverages Cartesian impedance control and a two-stage data collection to produce robot-only demonstrations, which are learned via diffusion policies. Empirical results across six tasks show substantial gains: force-informed actions achieve 76% average success, while force-agnostic baselines underperform; including force observations in policy inputs further boosts performance, particularly for tightly coordinated tasks. The work emphasizes the value of incorporating haptic information into imitation learning and outlines directions for scalable data collection and enhanced tactile feedback in future research.

Abstract

Imitation learning requires high-quality demonstrations consisting of sequences of state-action pairs. For contact-rich dexterous manipulation tasks that require dexterity, the actions in these state-action pairs must produce the right forces. Current widely-used methods for collecting dexterous manipulation demonstrations are difficult to use for demonstrating contact-rich tasks due to unintuitive human-to-robot motion retargeting and the lack of direct haptic feedback. Motivated by these concerns, we propose DexForce. DexForce leverages contact forces, measured during kinesthetic demonstrations, to compute force-informed actions for policy learning. We collect demonstrations for six tasks and show that policies trained on our force-informed actions achieve an average success rate of 76% across all tasks. In contrast, policies trained directly on actions that do not account for contact forces have near-zero success rates. We also conduct a study ablating the inclusion of force data in policy observations. We find that while using force data never hurts policy performance, it helps most for tasks that require advanced levels of precision and coordination, like opening an AirPods case and unscrewing a nut.

DexForce: Extracting Force-informed Actions from Kinesthetic Demonstrations for Dexterous Manipulation

TL;DR

DexForce introduces force-informed actions derived from kinesthetic demonstrations by augmenting observed fingertip positions with measured contact forces, enabling effective imitation learning for contact-rich dexterous manipulation. The method leverages Cartesian impedance control and a two-stage data collection to produce robot-only demonstrations, which are learned via diffusion policies. Empirical results across six tasks show substantial gains: force-informed actions achieve 76% average success, while force-agnostic baselines underperform; including force observations in policy inputs further boosts performance, particularly for tightly coordinated tasks. The work emphasizes the value of incorporating haptic information into imitation learning and outlines directions for scalable data collection and enhanced tactile feedback in future research.

Abstract

Imitation learning requires high-quality demonstrations consisting of sequences of state-action pairs. For contact-rich dexterous manipulation tasks that require dexterity, the actions in these state-action pairs must produce the right forces. Current widely-used methods for collecting dexterous manipulation demonstrations are difficult to use for demonstrating contact-rich tasks due to unintuitive human-to-robot motion retargeting and the lack of direct haptic feedback. Motivated by these concerns, we propose DexForce. DexForce leverages contact forces, measured during kinesthetic demonstrations, to compute force-informed actions for policy learning. We collect demonstrations for six tasks and show that policies trained on our force-informed actions achieve an average success rate of 76% across all tasks. In contrast, policies trained directly on actions that do not account for contact forces have near-zero success rates. We also conduct a study ablating the inclusion of force data in policy observations. We find that while using force data never hurts policy performance, it helps most for tasks that require advanced levels of precision and coordination, like opening an AirPods case and unscrewing a nut.
Paper Structure (5 sections, 3 equations, 9 figures)

This paper contains 5 sections, 3 equations, 9 figures.

Table of Contents

  1. Introduction
  2. Related work
  3. Method
  4. Experiments
  5. Conclusion

Figures (9)

  • Figure 1: (a) DexForce extracts force-informed actions (orange squares) from kinesthetic demonstrations by augmenting observed robot positions (blue circles) according to contact forces (orange arrows) measured with 6-axis force-torque sensors (green). (b) Executing force-informed actions allows the robot to reproduce forces applied in the kinesthetic demonstration, thereby executing the task. DexForce enables us to collect high-quality demonstrations for training policies on a variety of contact-rich tasks like (c) unscrewing a nut, (d) opening an AirPods case and (e) flipping a box.
  • Figure 2: The left column shows kinesthetic demonstrations where the operator applies (a) a small force and (b) a larger force to a surface. In both scenarios, even though the operator applies different amounts of force, the current fingertip position recorded during the demonstrations, denoted with a blue circle, is the same. The difference between the observed fingertip positions and force-informed targets is proportional to the contact force $f$ (c, d). Note: we visualize $-f$, the contact force exerted by the object onto the finger.
  • Figure 3: The DexForce two-stage demonstration collection procedure, illustrated with a one-finger task where the robot must slide a purple square along a fixed surface to the red goal region. Note: we visualize $-f$, the contact force exerted by the object onto the finger.
  • Figure 4: Task names, descriptions, example trajectories, success criteria, and number of training demonstrations.
  • Figure 5: In our demonstrations, we randomly initialize object poses within the green regions. For our in-distribution evaluations, we sample 30 initial poses from within these green regions. For the unscrew nut demonstrations, we fix the position of the mount and randomize the initial rotation of the nut between 0°-360°. OOD scenarios: For unscrew nut, initial rotation of nut between 360°-900°. For flip box, the box is 100g heavier.
  • ...and 4 more figures