Glovity: Learning Dexterous Contact-Rich Manipulation via Spatial Wrench Feedback Teleoperation System
Yuyang Gao, Haofei Ma, Pai Zheng
TL;DR
Glovity tackles the lack of multimodal feedback and embodiment gaps in dexterous teleoperation by integrating a wearable spatial wrench feedback device with a fingertip-calibrated haptic glove, enabling intuitive force-torque and tactile cues at low cost. The system employs a modular, open-source design, combining a palm-mounted wrench feedback mechanism, a Hall-calibrated haptic glove, and Vive-based hand tracking to enable efficient retargeting to dexterous robots. Experimental results show improved task performance in wrench-assisted teleoperation and thin-object grasping, and demonstrate that wrench signals can enhance diffusion-based imitation learning (DP-R3M) for novel contact-rich scenarios. These findings indicate Glovity’s potential to enable scalable, data-efficient learning and human-in-the-loop control for complex manipulation tasks in real-world robotics.
Abstract
We present Glovity, a novel, low-cost wearable teleoperation system that integrates a spatial wrench (force-torque) feedback device with a haptic glove featuring fingertip Hall sensor calibration, enabling feedback-rich dexterous manipulation. Glovity addresses key challenges in contact-rich tasks by providing intuitive wrench and tactile feedback, while overcoming embodiment gaps through precise retargeting. User studies demonstrate significant improvements: wrench feedback boosts success rates in book-flipping tasks from 48% to 78% and reduces completion time by 25%, while fingertip calibration enhances thin-object grasping success significantly compared to commercial glove. Furthermore, incorporating wrench signals into imitation learning (via DP-R3M) achieves high success rate in novel contact-rich scenarios, such as adaptive page flipping and force-aware handovers. All hardware designs, software will be open-sourced. Project website: https://glovity.github.io/