Exo-ViHa: A Cross-Platform Exoskeleton System with Visual and Haptic Feedback for Efficient Dexterous Skill Learning

TL;DR

Exo-ViHa addresses data collection bottlenecks in dexterous imitation learning by integrating a first-person, haptic exoskeleton with cross-platform compatibility. The approach fuses SLAM-based end-effector tracking, motion-capture hand data, and multi-view visual input, trained with ACT to produce deployment-ready dexterous control. Experiments show near-human data collection efficiency and about 80% success in real robot deployment across multi-contact tasks, outperforming teleoperation in key aspects. This work offers practical impact for rapid, realistic skill acquisition in robotic manipulation.

Abstract

Imitation learning has emerged as a powerful paradigm for robot skills learning. However, traditional data collection systems for dexterous manipulation face challenges, including a lack of balance between acquisition efficiency, consistency, and accuracy. To address these issues, we introduce Exo-ViHa, an innovative 3D-printed exoskeleton system that enables users to collect data from a first-person perspective while providing real-time haptic feedback. This system combines a 3D-printed modular structure with a slam camera, a motion capture glove, and a wrist-mounted camera. Various dexterous hands can be installed at the end, enabling it to simultaneously collect the posture of the end effector, hand movements, and visual data. By leveraging the first-person perspective and direct interaction, the exoskeleton enhances the task realism and haptic feedback, improving the consistency between demonstrations and actual robot deployments. In addition, it has cross-platform compatibility with various robotic arms and dexterous hands. Experiments show that the system can significantly improve the success rate and efficiency of data collection for dexterous manipulation tasks.

Figures (6)

• Figure 1: Overview of the Exo-ViHa system. (a) Demonstration of the exoskeleton-based data collection system. (b) Data collection of diverse tasks in various scenarios. (c) Compatibility of the Exo-ViHa system with different dexterous robotic hands and arms.
• Figure 2: Details of the exoskeleton system. (a) The components of the exoskeleton system. (b) Exoskeleton wearing demonstration.
• Figure 3: Details of the Exo-ViHa system. (a) Demonstration of the exoskeleton used for data collection and schematic diagram of the first-person perspective and haptic feedback. (b) Illustration of the working space range. (c) The visual information captured by the external cameras and the wrist camera is basically consistent during both the data collection and deployment phases. (d) The power source and microcomputer are placed into a backpack, allowing the system to be used for data collection in outdoor environments.
• Figure 4: Workflow of data collection and deployment of Exo-ViHa. The collected data include end-effector states, 6D end-effector pose and camera frames.
• Figure 5: Data collection efficiency comparision across different scenarios. (a-b) Comparative analysis of whiteboard erasing tasks: (a) Multi-contact interaction scenarios demonstrating human action, Exo-ViHa and teleoperation; (b) Quantitative time efficiency comparison across methods. (c) Pick and place scenario demonstrating the execution process for teleoperation, Exo-ViHa, and human action.