Bi-directional Momentum-based Haptic Feedback and Control System for In-Hand Dexterous Telemanipulation

TL;DR

Bi-Hap tackles the lack of real-time torque feedback in in-hand dexterous telemanipulation by introducing a palm-sized momentum-based haptic device with an IMU and a learning-enabled closed-loop controller. It combines a flywheel-based actuator, impedance and velocity control, and an error-adaptive feedback strategy to render torque and vibration without external grounding. The approach achieves sub-0.025 s command-following latency and torque RMSE under 0.010 Nm, outperforming comparable ungrounded devices, and demonstrably improves operator performance in both offline torque fidelity tests and online telemanipulation tasks. This work enables portable, bidirectional haptic feedback for fine in-hand manipulation and lays groundwork for extending to 3-DoF torque feedback on real robotic hands.

Abstract

In-hand dexterous telemanipulation requires not only precise remote motion control of the robot but also effective haptic feedback to the human operator to ensure stable and intuitive interactions between them. Most existing haptic devices for dexterous telemanipulation focus on force feedback and lack effective torque rendering, which is essential for tasks involving object rotation. While some torque feedback solutions in virtual reality applications-such as those based on geared motors or mechanically coupled actuators-have been explored, they often rely on bulky mechanical designs, limiting their use in portable or in-hand applications. In this paper, we propose a Bi-directional Momentum-based Haptic Feedback and Control (Bi-Hap) system that utilizes a palm-sized momentum-actuated mechanism to enable real-time haptic and torque feedback. The Bi-Hap system also integrates an Inertial Measurement Unit (IMU) to extract the human's manipulation command to establish a closed-loop learning-based telemanipulation framework. Furthermore, an error-adaptive feedback strategy is introduced to enhance operator perception and task performance in different error categories. Experimental evaluations demonstrate that Bi-Hap achieved feedback capability with low command following latency (Delay < 0.025 s) and highly accurate torque feedback (RMSE < 0.010 Nm).

Figures (6)

• Figure 1: The data flow in the proposed Bi-Hap system establishes a bi-directional feedback and control loop, enabling not only DRL-based teleoperation from the human to the robot but also recreating a seamless manipulation experience through torque and vibration feedback delivered from the robot back to the human.
• Figure 2: System architecture of the Bi-Hap system, illustrating key modules for control, sensing, power, and torque feedback.
• Figure 3: Exploded view of the Bi-Hap device with three-layer cubic structure (side length = 60 mm), for easy maintenance, component replacement, and intuitive in-hand telemanipulation.
• Figure 4: Experimental setup for: (a) human offline test; (b) human online test. The white area in Fig. 4(b) represents the target position, and points are awarded when the red dot (represents the operator's current position) enters this area, corresponding to Target Position Reached. Exceeding the boundary of the white area corresponds to Target Position Overshot. The blue area serves as a warning zone, reminding the operator that adjustments are needed, corresponding to Normal Manipulation. The green area represents the forbidden zone, corresponding to Far From Target Position.
• Figure 5: Tracking performance of (a) sinusoidal and (b) square Goal.