From Vision to Assistance: Gaze and Vision-Enabled Adaptive Control for a Back-Support Exoskeleton

TL;DR

The paper addresses the need for timely, context-aware back-support exoskeleton assistance in industrial tasks by introducing a vision-gated control framework that fuses egocentric vision, gaze tracking, and a YOLOv9-based grasp detector with a finite-state policy and a variable admittance controller. The approach enables load- and task-aware torque delivery, improving responsiveness, safety, and user experience. A 15-participant study in a stooping box-handling task shows vision-enabled assistance reduces perceived physical demand and enhances fluency, trust, and comfort, with robust perception performance (Box-Grasped precision 96.12% etc.). The results underscore the potential of perception-guided augmentation for ergonomic and safety improvements in industrial wearable robotics, and point to future work on varied payloads and integrated joint control.

Abstract

Back-support exoskeletons have been proposed to mitigate spinal loading in industrial handling, yet their effectiveness critically depends on timely and context-aware assistance. Most existing approaches rely either on load-estimation techniques (e.g., EMG, IMU) or on vision systems that do not directly inform control. In this work, we present a vision-gated control framework for an active lumbar occupational exoskeleton that leverages egocentric vision with wearable gaze tracking. The proposed system integrates real-time grasp detection from a first-person YOLO-based perception system, a finite-state machine (FSM) for task progression, and a variable admittance controller to adapt torque delivery to both posture and object state. A user study with 15 participants performing stooping load lifting trials under three conditions (no exoskeleton, exoskeleton without vision, exoskeleton with vision) shows that vision-gated assistance significantly reduces perceived physical demand and improves fluency, trust, and comfort. Quantitative analysis reveals earlier and stronger assistance when vision is enabled, while questionnaire results confirm user preference for the vision-gated mode. These findings highlight the potential of egocentric vision to enhance the responsiveness, ergonomics, safety, and acceptance of back-support exoskeletons.

Figures (6)

• Figure 1: User performing the lifting task with the back–support exoskeleton (blue arrow). The inset (bottom right) provides the egocentric perspective from the gaze–tracking glasses (orange arrow), where the target box is highlighted in green box and the user’s gaze fixation is marked in red circle.
• Figure 2: System model. The forces and torques acting on the exoskeleton and the user during a handling task.
• Figure 3: The Vision Gate integrates gaze and object detection to identify grasp events. This information is passed to a Finite–State Machine (FSM) that governs the assistance logic depending on the user’s posture. A Variable Admittance Controller adapts torque output $\tau_{ass}$ and impedance parameters ($K,C$) according to the current state. An embedded hardware processes the control signal driving the back-support exoskeleton, assisting the user during lifting tasks.
• Figure 4: Examples of egocentric frames from the Tobii Pro Glasses 3. Green bounding boxes with labels indicate YOLO detections, while the red circle marks the gaze point.
• Figure 5: Mean questionnaire scores under the three experimental conditions. Error bars show standard deviations.