A Multipurpose Interface for Close- and Far-Proximity Control of Mobile Collaborative Robots

TL;DR

The paper tackles the challenge of enabling intuitive, non-expert control of Mobile Collaborative Robots across near and far contexts by introducing the Open Multi-Purpose Interface (OMI), a passive detachable device that provides local haptic control when attached and VIO-based teleoperation when detached. It combines a haptic handle (M5Core2) with a handheld visual system and a whole-body Cartesian impedance controller to coordinate manipulation and locomotion, mapping interface motions to the robot through precise transforms and virtual wrenches. The authors evaluate multiple VIO configurations, demonstrating that a wireless stereo setup offers comparable or better accuracy and update rates than RGB-D configurations, while a home-care study shows higher task success with the VIO interface and similar workload to traditional Xsens-based teleoperation. The results support the proposed interface as a cost-effective, calibration-free, and versatile solution for remote and in-situ MCR operation, with clear paths toward integration with learning from demonstrations and shared autonomy.

Abstract

This letter introduces an innovative visuo-haptic interface to control Mobile Collaborative Robots (MCR). Thanks to a passive detachable mechanism, the interface can be attached/detached from a robot, offering two control modes: local control (attached) and teleoperation (detached). These modes are integrated with a robot whole-body controller and presented in a unified close- and far-proximity control framework for MCR. The earlier introduction of the haptic component in this interface enabled users to execute intricate loco-manipulation tasks via admittance-type control, effectively decoupling task dynamics and enhancing human capabilities. In contrast, this ongoing work proposes a novel design that integrates a visual component. This design utilizes Visual-Inertial Odometry (VIO) for teleoperation, estimating the interface's pose through stereo cameras and an Inertial Measurement Unit (IMU). The estimated pose serves as the reference for the robot's end-effector in teleoperation mode. Hence, the interface offers complete flexibility and adaptability, enabling any user to operate an MCR seamlessly without needing expert knowledge. In this letter, we primarily focus on the new visual feature, and first present a performance evaluation of different VIO-based methods for teleoperation. Next, the interface's usability is analyzed in a home-care application and compared to an alternative designed by a commercial MoCap system. Results show comparable performance in terms of accuracy, completion time, and usability. Nevertheless, the proposed interface is low-cost, poses minimal wearability constraints, and can be used anywhere and anytime without needing external devices or additional equipment, offering a versatile and accessible solution for teleoperation.

Figures (7)

• Figure 1: The proposed VIO-based teleoperation interface enables individuals to control MCRs in both close and distant settings, facilitating the performance of Activities of Daily Living (ADLs) in home care environments.
• Figure 2: Block diagram of the extended visuo-haptic framework, to be used for MCRs under a whole-body impedance controller. Dashed lines represent connections related to the haptic part whilst solid lines represent those related to the visual part.
• Figure 3: The image showcases the designed model of the interface on the left, with the corresponding assembled version of the interface presented on the right.
• Figure 4: A sketch of the experimental setup, used for home-care scenario application.
• Figure 5: This figure showcases the VIO performance using two sensors, Stereo and RGB-D, in two setups: wired (left column labeled as (a)) and wireless (right column labeled as (b)). The bottom plots in both columns display the absolute error in position estimation, with the red line representing the RGB-D and the blue line representing the stereo setup.