A Laser-guided Interaction Interface for Providing Effective Robot Assistance to People with Upper Limbs Impairments

TL;DR

The paper addresses assistive robotics for people with upper-limb impairments by introducing a head-mounted laser interface that guides a robotic arm to target locations and manipulate objects. It combines two control modalities—a direct environment targeting mode and a virtual keyboard mode—that enable both co-manipulation and end-effector control. A neural-network-based laser spot detector on RGB-D data enables real-time command updates at about $30\, \mathrm{Hz}$, while the robot relies on MoveIt for collision-free trajectory planning and CartesI/O/XBot2 for control. Experiments with healthy subjects simulating impairments show faster, more precise performance and reduced head movement compared to IMU-based interfaces, and demonstrate applicability to bread cutting, bottle opening, and pick-and-place tasks, supporting ADL independence.

Abstract

Robotics has shown significant potential in assisting people with disabilities to enhance their independence and involvement in daily activities. Indeed, a societal long-term impact is expected in home-care assistance with the deployment of intelligent robotic interfaces. This work presents a human-robot interface developed to help people with upper limbs impairments, such as those affected by stroke injuries, in activities of everyday life. The proposed interface leverages on a visual servoing guidance component, which utilizes an inexpensive but effective laser emitter device. By projecting the laser on a surface within the workspace of the robot, the user is able to guide the robotic manipulator to desired locations, to reach, grasp and manipulate objects. Considering the targeted users, the laser emitter is worn on the head, enabling to intuitively control the robot motions with head movements that point the laser in the environment, which projection is detected with a neural network based perception module. The interface implements two control modalities: the first allows the user to select specific locations directly, commanding the robot to reach those points; the second employs a paper keyboard with buttons that can be virtually pressed by pointing the laser at them. These buttons enable a more direct control of the Cartesian velocity of the end-effector and provides additional functionalities such as commanding the action of the gripper. The proposed interface is evaluated in a series of manipulation tasks involving a 6DOF assistive robot manipulator equipped with 1DOF beak-like gripper. The two interface modalities are combined to successfully accomplish tasks requiring bimanual capacity that is usually affected in people with upper limbs impairments.

Figures (13)

• Figure 1: The laser-guided interface provides assistance to users with arm impairments. On the left, the user is commanding a target location, to make the robot hold the bread while he will cut it with his healthy arm. On the right, the user is commanding a yaw rotation with the keyboard to unscrew the cap of the bottle.
• Figure 2: Scheme representing the overall interface. Depending on where the laser is pointed (environment or keyboard), the relative control modality is exploited to generate robot commands.
• Figure 3: Scheme representing the laser detection pipeline. The RGB-D camera provides RGB images and point clouds. From the RGB image, the neural network extracts the 2D pixel coordinates of the laser spot. These coordinates are then matched with the point cloud to extract the 3D position of the laser.
• Figure 4: Experimental setup. On the left, the user points the laser to a goal. On the top-right, RViz displays the laser spot and the obstacles considered for generating the trajectory to the goal. On the bottom-right, the laser spot is detected by the system.
• Figure 5: Our laser-based interface is compared with a IMU-based interface in a reaching targets task.