A novel gesture interaction control method for rehabilitation lower extremity exoskeleton
Shuang Qiu, Zhongcai Pei, Chen Wang, Jing Zhang, Zhiyong Tang
TL;DR
The paper addresses discomfort and unreliability in contact-based HRI for rehabilitation exoskeletons by introducing a non-contact gesture control framework using a monocular RGB camera. The proposed KG DFA approach integrates hand keypoint detection, a two-layer convex gesture recognition, and depth-triggered augmented reality triggering, coupled with an FSM to manage safe gait transitions. It achieves high gesture recognition performance (mAP@0.5 ≈ 95.7%, many gestures >95% AP) and real-time responsiveness (average ART ≈ 0.615 s in overall operation, US ≈ 0.03 s per frame, ~34 FPS), demonstrating feasibility for real-time control of RLEEX. The work highlights improvements in comfort and intuitive interaction while acknowledging environmental and latency limitations, and suggests future multi-sensor fusion to enhance robustness and applicability.
Abstract
With the rapid development of Rehabilitation Lower Extremity Robotic Exoskeletons (RLEEX) technology, significant advancements have been made in Human-Robot Interaction (HRI) methods. These include traditional physical HRI methods that are easily recognizable and various bio-electrical signal-based HRI methods that can visualize and predict actions. However, most of these HRI methods are contact-based, facing challenges such as operational complexity, sensitivity to interference, risks associated with implantable devices, and, most importantly, limitations in comfort. These challenges render the interaction less intuitive and natural, which can negatively impact patient motivation for rehabilitation. To address these issues, this paper proposes a novel non-contact gesture interaction control method for RLEEX, based on RGB monocular camera depth estimation. This method integrates three key steps: detecting keypoints, recognizing gestures, and assessing distance, thereby applying gesture information and augmented reality triggering technology to control gait movements of RLEEX. Results indicate that this approach provides a feasible solution to the problems of poor comfort, low reliability, and high latency in HRI for RLEEX platforms. Specifically, it achieves a gesture-controlled exoskeleton motion accuracy of 94.11\% and an average system response time of 0.615 seconds through non-contact HRI. The proposed non-contact HRI method represents a pioneering advancement in control interactions for RLEEX, paving the way for further exploration and development in this field.