Eye-tracking-Driven Shared Control for Robotic Arms:Wizard of Oz Studies to Assess Design Choices

TL;DR

This work tackles the challenge of enabling intuitive control of assistive robotic arms for people with physical disabilities by combining eye-tracking with a shared-control paradigm. A Wizard of Oz design allowed rapid evaluation of how gaze-driven actions could automate tasks such as serving food, turning on lights, and environmental interactions, without implementing full autonomy. Through an online survey of stakeholders and a hands-on study with gaze data, the authors identify key usability factors, task priorities, and technical constraints (e.g., dwell-time thresholds, object recognition accuracy, and safety). The findings inform design adaptations toward robust gaze-detection, adjustable robot velocity, and reliable task-automation feedback, aiming to improve accessibility and independence in daily life for users with limited motor function. Overall, the work demonstrates how early, community-informed, Wizard-of-Oz testing can steer the development of practical eye-tracking shared-control systems for assistive robotics while highlighting practical challenges to address in future iterations.

Abstract

Advances in eye-tracking control for assistive robotic arms provide intuitive interaction opportunities for people with physical disabilities. Shared control has gained interest in recent years by improving user satisfaction through partial automation of robot control. We present an eye-tracking-guided shared control design based on insights from state-of-the-art literature. A Wizard of Oz setup was used in which automation was simulated by an experimenter to evaluate the concept without requiring full implementation. This approach allowed for rapid exploration of user needs and expectations to inform future iterations. Two studies were conducted to assess user experience, identify design challenges, and find improvements to ensure usability and accessibility. The first study involved people with disabilities by providing a survey, and the second study used the Wizard of Oz design in person to gain technical insights, leading to a comprehensive picture of findings.

Figures (8)

• Figure 1: Wizard of Oz design: The participant selects a task by focusing his or her gaze on the desired object. The gaze is visualized on the computer by displaying the live-video of the eye-tracker. The experimenter starts the task via the robot's web interface. The robot then performs the task and returns to the home position to wait for further commands. The home position is used as indication that the participant can select the next task.
• Figure 2: Nine interactable objects were presented with which five tasks were performed. Participants were shielded from visual distractions by green walls surrounding the experimental area.
• Figure 3: Distribution of questions asked for each group. Q outlines the questions' context and the number of the questions for each category is given.
• Figure 4: Distribution of responses from online survey participants. Q1 to Q3 were answered by participants with disabilities, Q4 to Q6 by family and friends, and Q7 and Q8 from healthcare professionals.
• Figure 5: Heat map visualizing the attention of participants during task selection (absolute duration), mapped to the snapshot of the experimental setup. Red areas indicate frequent fixations, green less fixations. Areas without coloring indicate no fixations within the observed intervals. Dashed outlines were added manually to better distinguish the edges of the hotspots.