Prometheus: Universal, Open-Source Mocap-Based Teleoperation System with Force Feedback for Dataset Collection in Robot Learning

TL;DR

Prometheus tackles the lack of force feedback in mocap-based teleoperation for dataset collection in robot learning. It introduces a low-cost, open-source system that uses HTC Vive Trackers 2.0, a UR3 arm, and a Robotiq gripper with a force-sensing finger to provide real-time force feedback for teleoperation and data collection. The study demonstrates that force feedback reduces grip force by 35.77% and enables Vision-Language-Action models to reach up to 90% success on deformable objects, with a 300-trajectory dataset across three tasks. By releasing open-source hardware and software, Prometheus offers a scalable, accessible solution for high-quality, force-aware imitation learning datasets.

Abstract

This paper presents a novel teleoperation system with force feedback, utilizing consumer-grade HTC Vive Trackers 2.0. The system integrates a custom-built controller, a UR3 robotic arm, and a Robotiq gripper equipped with custom-designed fingers to ensure uniform pressure distribution on an embedded force sensor. Real-time compression force data is transmitted to the controller, enabling operators to perceive the gripping force applied to objects. Experimental results demonstrate that the system enhances task success rates and provides a low-cost solution for large-scale imitation learning data collection without compromising affordability.

Figures (12)

• Figure 1: Main view of the system
• Figure 2:
• Figure 3: Principle of work
• Figure 4: Internal schematic of the controller
• Figure 5: Gripper with a force-feedback robotic finger