Just Add Force for Contact-Rich Robot Policies

TL;DR

It is found that forceful policies are superior to position-only policies for delicate grasping and are able to generalize to unseen delicate objects, while reducing grasp policy latency by near 4x, relative to LLM-based methods.

Abstract

Robot trajectories used for learning end-to-end robot policies typically contain end-effector and gripper position, workspace images, and language. Policies learned from such trajectories are unsuitable for delicate grasping, which require tightly coupled and precise gripper force and gripper position. We collect and make publically available 130 trajectories with force feedback of successful grasps on 30 unique objects. Our current-based method for sensing force, albeit noisy, is gripper-agnostic and requires no additional hardware. We train and evaluate two diffusion policies: one with (forceful) the collected force feedback and one without (position-only). We find that forceful policies are superior to position-only policies for delicate grasping and are able to generalize to unseen delicate objects, while reducing grasp policy latency by near 4x, relative to LLM-based methods. With our promising results on limited data, we hope to signal to others to consider investing in collecting force and other such tactile information in new datasets, enabling more robust, contact-rich manipulation in future robot foundation models. Our data, code, models, and videos are viewable at https://justaddforce.github.io/.

Figures (6)

• Figure 1: We leverage LLM-directed expert demonstrations dg of delicate objects to generate a dataset of 130 successful grasps of 30 different objects spanning a variety of physical properties. Our trajectories, unlike other datasets used in end-to-end learning openxdroid, contain observed gripper applied and contact force and the action of increased gripper applied force. We train diffusion policies dp on the dataset with and without force data and observe that forceful policies can, despite limited data, replicate trained behavior and generalize to unseen delicate objects at 4x reduced latency relative to LLM-policies, and position-only policies cannot.
• Figure 2: We conduct a series of 10 trials for a selection of 10 objects; four seen in training, six unseen. Forceful policies (82%) replicate seen grasps (85%) and generalize to similar but unseen objects (80%). position-only policies (54%) retain a level of performance on seen (45%) and improve on unseen (60%) delicate objects, suggesting that continuous gripper position control alone contributes to successful delicate grasps. We note that position-only policy failures are generally deforming and compress more than forceful policies (see Fig. \ref{['fig:breakdown']})
• Figure 3: We plot 1) forceful policies gripper position (blue), applied force (green dash), and contact force (purple dash) and 2) position-only policies gripper position (red) against time, with additional plots in \ref{['appendix:extra_unseen']}. Uniformly, position-only policies close more narrowly than forceful policies, leading to deformation failures, particularly for delicate objects like blackberries and raspberries. Individual position-only policy grasps on produce like tomatoes and peppers are successful, but we observe that after 10 trials, the produce is noticeably deformed due to greater compression, unlike for forceful grasps. On objects like the pepper, empty taco, blackberry, and tomato, applied force flattens as contact force increases.
• Figure :
• Figure :