Independence in the Home: A Wearable Interface for a Person with Quadriplegia to Teleoperate a Mobile Manipulator

TL;DR

This work addresses restoring independence for people with severe motor impairments by teleoperating a mobile manipulator via HAT, an IMU-based head-worn interface. It introduces Driver Assistance as a perception-guided shared-control mechanism and a GUI-enabled off-line pathway with open-vocabulary perception, demonstrated in a 7-day in-home study with a non-speaking quadriplegic, showing improvements in task completion time, error rate, and workload while preserving user control. The paper contributes a full system design (hardware, mode switching, and perception), a formal modeling and control framework for shared autonomy, and empirical evidence of viability, customizability, and potential for home adoption. The results highlight practical impact for home assistance, while outlining safety considerations and future directions for broader testing and integration with screen-based or autonomous capabilities, including drift handling and anomaly detection.

Abstract

Teleoperation of mobile manipulators within a home environment can significantly enhance the independence of individuals with severe motor impairments, allowing them to regain the ability to perform self-care and household tasks. There is a critical need for novel teleoperation interfaces to offer effective alternatives for individuals with impairments who may encounter challenges in using existing interfaces due to physical limitations. In this work, we iterate on one such interface, HAT (Head-Worn Assistive Teleoperation), an inertial-based wearable integrated into any head-worn garment. We evaluate HAT through a 7-day in-home study with Henry Evans, a non-speaking individual with quadriplegia who has participated extensively in assistive robotics studies. We additionally evaluate HAT with a proposed shared control method for mobile manipulators termed Driver Assistance and demonstrate how the interface generalizes to other physical devices and contexts. Our results show that HAT is a strong teleoperation interface across key metrics including efficiency, errors, learning curve, and workload. Code and videos are located on our project website.

Figures (9)

• Figure 1: Three robot control modes: Drive, Arm, Wrist and mapping of head orientation to robot motion in arm mode.
• Figure 2: A subset of the tasks conducted during the study. Pertinent objects and tools are highlighted in orange.
• Figure 3: Left: Completion time and resets for the repeated blanket+Tissue+Trash task. A best fit line is plotted, showing a downward trend in completion times over the course of the study. Right: Completion time and resets for all other predefined tasks. Improvement is seen in task completion times and resets required for all repeated tasks over the course of the study.
• Figure 4: 7-point Likert item responses for Ease of Use, No Errors, Error Recovery, Reasonable Time, and Preference for each day of the study. For all items, a higher rating is better.
• Figure 5: 7-point NASA TLX responses for each day of the study. For all items, a lower rating is better.