Sensory Glove-Based Surgical Robot User Interface

TL;DR

This work tackles the bulky, space-consuming nature of standard robotic-surgery consoles by introducing a glove-based interface that uses a Manus Meta Prime 3 XR glove, an HTC Vive Tracker, and SCOPEYE glasses to control a single arm of the da Vinci system. It combines pose and finger tracking with gesture recognition and a novel clutch that also adjusts instrument orientation, augmented by vibrotactile feedback to keep the surgeon informed without cluttering the visual field. The methodology covers system architecture, calibrated kinematics, workspace scaling, and multiple gesture-classification approaches, with quantitative and qualitative evaluations demonstrating high gesture-recognition accuracy and comparable task performance to the conventional console, at a fraction of footprint and cost (approximately $5000). The results indicate strong potential for compact, adaptable surgical interfaces and point to future work including velocity control, full two-arm control, and ergonomic enhancements to further improve usability and efficiency in the OR.

Abstract

Robotic surgery has reached a high level of maturity and has become an integral part of standard surgical care. However, existing surgeon consoles are bulky, take up valuable space in the operating room, make surgical team coordination challenging, and their proprietary nature makes it difficult to take advantage of recent technological advances, especially in virtual and augmented reality. One potential area for further improvement is the integration of modern sensory gloves into robotic platforms, allowing surgeons to control robotic arms intuitively with their hand movements. We propose one such system that combines an HTC Vive tracker, a Manus Meta Prime 3 XR sensory glove, and SCOPEYE wireless smart glasses. The system controls one arm of a da Vinci surgical robot. In addition to moving the arm, the surgeon can use fingers to control the end-effector of the surgical instrument. Hand gestures are used to implement clutching and similar functions. In particular, we introduce clutching of the instrument orientation, a functionality unavailable in the da Vinci system. The vibrotactile elements of the glove are used to provide feedback to the user when gesture commands are invoked. A qualitative and quantitative evaluation has been conducted that compares the current device with the dVRK console. The system is shown to have excellent tracking accuracy, and the new interface allows surgeons to perform common surgical training tasks with minimal practice efficiently.

Figures (3)

• Figure 1: The setup showing the configuration of the da Vinci arm (PSM) and the Manus glove with the HTC Vive Tracker attached. The user wears the SCOPEYE smart glasses for the stereo endoscopic view. Shown are the $x$ (red), $y$ (green), and $z$ (blue) axes of the relevant coordinate frames.
• Figure 2: A sample trajectory of the glove and the PSM: (a) translation; (b) orientation; and (c) the distance of the thumb and index fingers on the glove and the jaw angle of the PSM. The gray and purple regions correspond to the periods when Ring and Clutch gestures, respectively, are active.
• Figure 3: The x-axis translation of the glove and the PSM tip: raw data (top) and data after the delay has been eliminated (bottom).