AdaptiX -- A Transitional XR Framework for Development and Evaluation of Shared Control Applications in Assistive Robotics

TL;DR

This paper tackles the usability gap in assistive robotics by providing AdaptiX, an open XR framework for developing and evaluating shared control in a high-resolution VR/MR environment with a Kinova Jaco 2 model and ROS integration. It centers the Adaptive DoF Mapping Control (ADMC) as a core strategy, offering CNN-based and script-based mappings and a mechanism to communicate suggestions via continuous or threshold-driven cues. The framework supports a full MR continuum, modular interfaces, and a versatile recording/replay system, demonstrated across three framework adaptations that explore interaction design, attention guidance, and input-device comparisons. The open-source nature and digital-twin/physical-twin readiness position AdaptiX as a practical, scalable tool for rapid ideation, testing, and evaluation of shared-control strategies in assistive robotics, with broad implications for HRI research and development.

Abstract

With the ongoing efforts to empower people with mobility impairments and the increase in technological acceptance by the general public, assistive technologies, such as collaborative robotic arms, are gaining popularity. Yet, their widespread success is limited by usability issues, specifically the disparity between user input and software control along the autonomy continuum. To address this, shared control concepts provide opportunities to combine the targeted increase of user autonomy with a certain level of computer assistance. This paper presents the free and open-source AdaptiX XR framework for developing and evaluating shared control applications in a high-resolution simulation environment. The initial framework consists of a simulated robotic arm with an example scenario in Virtual Reality (VR), multiple standard control interfaces, and a specialized recording/replay system. AdaptiX can easily be extended for specific research needs, allowing Human-Robot Interaction (HRI) researchers to rapidly design and test novel interaction methods, intervention strategies, and multi-modal feedback techniques, without requiring an actual physical robotic arm during the early phases of ideation, prototyping, and evaluation. Also, a Robot Operating System (ROS) integration enables the controlling of a real robotic arm in a PhysicalTwin approach without any simulation-reality gap. Here, we review the capabilities and limitations of AdaptiX in detail and present three bodies of research based on the framework. AdaptiX can be accessed at https://adaptix.robot-research.de.

Figures (13)

• Figure 1: Overview of AdaptiX' architecture, illustrating each component, their directional communication, and the crossover from and to the framework. The user input is either used for Cartesian Control or ADMC. For ADMC, either a CNN-based or script-based rule engine can be selected.
• Figure 2: Suggestions as Visualized in the ADMC, (a) Continue previous movement, (b) Optimal Suggestion, (c) Adjustment Suggestion, (d) Translation Suggestion, (e) Rotation Suggestion, (f) Gripper Suggestion. Colors: Bright cyan arrow: Currently active DoF mapping. Dark blue arrow: Next most likely DoF mapping.
• Figure 3: Concept of adaptive DoF mapping control. (a) Control pipeline for proposed adaptive shared control and (b) matrix representation of DoF mappings: Columns represent input-DoF. Rows represent output-DoF. Subsets represent modes. Two empty columns were added to represent zero movement mappings in Finger Mode.
• Figure 4: MR continuum with (a) only the real robotic arm in real environment, (b) augmenting of directional cues in the real environment with the real robotic arm, (c) additional visualizing the gripper and base of the virtual robotic arm in the real environment, (d) visualizing the simulated robotic arm in the real environment, (e) visualizing the real robotic arm in the virtual environment, and (f) visualizing the simulated robotic arm in the virtual environment.
• Figure 5: Component connections of the ROS interface for mixed reality.