Exploring AI-enhanced Shared Control for an Assistive Robotic Arm

TL;DR

This work addresses the problem that assistive robotic arms impose high manual control demands, yet users with motor impairments often seek autonomy; it motivates AI-enhanced shared control to preserve self-determination while reducing effort. The authors present the AdaptiXPascher XR testbed that couples a Kinova Jaco 2 with a 3D simulation and a DigitalTwin/PhysicalTwin architecture to enable AI-driven DoF mappings under user approval, within a MR/VR-enabled environment. They identify three core challenges—AI legibility, AI user control, and AI intervention—and propose visualization cues (e.g., $DoF$ indicators, gizmo visualizations, AR overlays) and interaction strategies that are evaluated in user studies. Preliminary results indicate AI-assisted mappings can reduce task time, mode switches, and workload while keeping users in the loop, but issues of trust, interpretability, and robust AI intervention remain critical for real-world deployment.

Abstract

Assistive technologies and in particular assistive robotic arms have the potential to enable people with motor impairments to live a self-determined life. More and more of these systems have become available for end users in recent years, such as the Kinova Jaco robotic arm. However, they mostly require complex manual control, which can overwhelm users. As a result, researchers have explored ways to let such robots act autonomously. However, at least for this specific group of users, such an approach has shown to be futile. Here, users want to stay in control to achieve a higher level of personal autonomy, to which an autonomous robot runs counter. In our research, we explore how Artifical Intelligence (AI) can be integrated into a shared control paradigm. In particular, we focus on the consequential requirements for the interface between human and robot and how we can keep humans in the loop while still significantly reducing the mental load and required motor skills.

Figures (9)

• Figure 1: Overview of AdaptiX' architecture, illustrating each component, their directional communication, and the crossover from and to the framework Pascher.2024adaptix.
• Figure 2: Virtual environment consisting of (left to right): a virtual canvas, the motion controller, a table with a blue object and red target, and a Kinova Jaco with an arrow-based visualization Pascher2023c.
• Figure 3: MR continuum with (a) only the real robotic arm in real environment, (b) augmenting of directional cues in the real environment with the real robotic arm, (c) additional visualizing the gripper and base of the virtual robotic arm in the real environment, (d) visualizing the simulated robotic arm in the real environment, (e) visualizing the real robotic arm in the virtual environment, and (f) visualizing the simulated robotic arm in the virtual environment Pascher.2024adaptix.
• Figure 4: DoF-Indicator: (a) LEDs directly attached at each robot's joint; (b) LEDs mounted on a bar in front of the robot referring to each joint (1--7) pascher.2022dof.
• Figure 5: DoF-Combination-Indicator: (a) as an AR overlay, supporting robot and visualization in line of sight; (b) as an icon in the screen's corner pascher.2022dof.