Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MR-UBi: Mixed Reality-Based Underwater Robot Arm Teleoperation System with Reaction Torque Indicator via Bilateral Control

Kohei Nishi, Masato Kobayashi, Yuki Uranishi

TL;DR

MR-UBi presents a cost-efficient, mixed reality–based underwater teleoperation system that integrates a four-channel bilateral controller with a novel Reaction Torque Indicator. By overlaying a real-time torque bar and hybrid hue on an MR-HMD, the system bridges haptic feedback and visual cues to improve grasping torque regulation. Experimental results show significant improvements in torque accuracy and usability (Lift and Pick&Place tasks) and reduced workload, demonstrating that visual–haptic integration enhances stability and operator confidence in underwater manipulation. The approach offers practical impact for safer, more precise underwater operations and scalable deployment due to its low per-arm cost and fast 1 kHz control loop.

Abstract

We present a mixed reality-based underwater robot arm teleoperation system with a reaction torque indicator via bilateral control (MR-UBi). The reaction torque indicator (RTI) overlays a color and length-coded torque bar in the MR-HMD, enabling seamless integration of visual and haptic feedback during underwater robot arm teleoperation. User studies with sixteen participants compared MR-UBi against a bilateral-control baseline. MR-UBi significantly improved grasping-torque control accuracy, increasing the time within the optimal torque range and reducing both low and high grasping torque range during lift and pick-and-place tasks with objects of different stiffness. Subjective evaluations further showed higher usability (SUS) and lower workload (NASA--TLX). Overall, the results confirm that \textit{MR-UBi} enables more stable, accurate, and user-friendly underwater robot-arm teleoperation through the integration of visual and haptic feedback. For additional material, please check: https://mertcookimg.github.io/mr-ubi

MR-UBi: Mixed Reality-Based Underwater Robot Arm Teleoperation System with Reaction Torque Indicator via Bilateral Control

TL;DR

MR-UBi presents a cost-efficient, mixed reality–based underwater teleoperation system that integrates a four-channel bilateral controller with a novel Reaction Torque Indicator. By overlaying a real-time torque bar and hybrid hue on an MR-HMD, the system bridges haptic feedback and visual cues to improve grasping torque regulation. Experimental results show significant improvements in torque accuracy and usability (Lift and Pick&Place tasks) and reduced workload, demonstrating that visual–haptic integration enhances stability and operator confidence in underwater manipulation. The approach offers practical impact for safer, more precise underwater operations and scalable deployment due to its low per-arm cost and fast 1 kHz control loop.

Abstract

We present a mixed reality-based underwater robot arm teleoperation system with a reaction torque indicator via bilateral control (MR-UBi). The reaction torque indicator (RTI) overlays a color and length-coded torque bar in the MR-HMD, enabling seamless integration of visual and haptic feedback during underwater robot arm teleoperation. User studies with sixteen participants compared MR-UBi against a bilateral-control baseline. MR-UBi significantly improved grasping-torque control accuracy, increasing the time within the optimal torque range and reducing both low and high grasping torque range during lift and pick-and-place tasks with objects of different stiffness. Subjective evaluations further showed higher usability (SUS) and lower workload (NASA--TLX). Overall, the results confirm that \textit{MR-UBi} enables more stable, accurate, and user-friendly underwater robot-arm teleoperation through the integration of visual and haptic feedback. For additional material, please check: https://mertcookimg.github.io/mr-ubi
Paper Structure (28 sections, 5 equations, 13 figures, 4 tables)

This paper contains 28 sections, 5 equations, 13 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Teleoperation in Land and Underwater Environments
  4. Haptic-Based Visual Feedback for Teleoperation
  5. MR-UBi: Mixed Reality-Based Underwater Robot Arm Teleoperation System with Reaction Torque Indicator via Bilateral Control
  6. Overview
  7. Underwater Robot Arm
  8. Bilateral Control
  9. Reaction Torque Indicator Design
  10. Experiments
  11. Experimental Design
  12. Preparation Phase (15 min).
  13. Task Phase (30 min × 2 method).
  14. Final Phase (5 min).
  15. Experimental Environment
  16. ...and 13 more sections

Figures (13)

  • Figure 1: Overview of Proposed MR-UBi
  • Figure 2: Overview of MR-UBi, Left: Follower System (Underwater), Right: Leader System (User), Center: Unity-based System.
  • Figure 3: Underwater Robot Arm (MR-UBi Arm)
  • Figure 4: Example of Bilateral Control Data (J4)
  • Figure 5: Design of Proposed Reaction Torque Indicator (RTI)
  • ...and 8 more figures