Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Real-Time Remote Control via VR over Limited Wireless Connectivity

H. P. Madushanka, Rafaela Scaciota, Sumudu Samarakoon, Mehdi Bennis

TL;DR

The paper addresses remote robot operation under limited wireless connectivity by proposing a VR-enabled interface that presents a real-time 3D map and enables seamless transition to autonomous navigation when links fail. It combines real-time SLAM-based mapping, VR teleoperation, connectivity monitoring, and autonomous path planning via the TurtleBot3 Navigation package, facilitated by a publish-subscribe data pipeline across two servers. The key contribution is an end-to-end demonstration platform that maintains task progress and reduces teleoperation lag during outages, demonstrated with a 2D gmapping-based map, VR visualization via a Varjo headset, and autonomous fallback. This work advances robust human-robot collaboration in bandwidth-constrained and dynamic environments with practical implications for field robotics and remote operation.

Abstract

This work introduces a solution to enhance human-robot interaction over limited wireless connectivity. The goal is toenable remote control of a robot through a virtual reality (VR)interface, ensuring a smooth transition to autonomous mode in the event of connectivity loss. The VR interface provides accessto a dynamic 3D virtual map that undergoes continuous updatesusing real-time sensor data collected and transmitted by therobot. Furthermore, the robot monitors wireless connectivity and automatically switches to a autonomous mode in scenarios with limited connectivity. By integrating four key functionalities: real-time mapping, remote control through glasses VR, continuous monitoring of wireless connectivity, and autonomous navigation during limited connectivity, we achieve seamless end-to-end operation.

Real-Time Remote Control via VR over Limited Wireless Connectivity

TL;DR

The paper addresses remote robot operation under limited wireless connectivity by proposing a VR-enabled interface that presents a real-time 3D map and enables seamless transition to autonomous navigation when links fail. It combines real-time SLAM-based mapping, VR teleoperation, connectivity monitoring, and autonomous path planning via the TurtleBot3 Navigation package, facilitated by a publish-subscribe data pipeline across two servers. The key contribution is an end-to-end demonstration platform that maintains task progress and reduces teleoperation lag during outages, demonstrated with a 2D gmapping-based map, VR visualization via a Varjo headset, and autonomous fallback. This work advances robust human-robot collaboration in bandwidth-constrained and dynamic environments with practical implications for field robotics and remote operation.

Abstract

This work introduces a solution to enhance human-robot interaction over limited wireless connectivity. The goal is toenable remote control of a robot through a virtual reality (VR)interface, ensuring a smooth transition to autonomous mode in the event of connectivity loss. The VR interface provides accessto a dynamic 3D virtual map that undergoes continuous updatesusing real-time sensor data collected and transmitted by therobot. Furthermore, the robot monitors wireless connectivity and automatically switches to a autonomous mode in scenarios with limited connectivity. By integrating four key functionalities: real-time mapping, remote control through glasses VR, continuous monitoring of wireless connectivity, and autonomous navigation during limited connectivity, we achieve seamless end-to-end operation.
Paper Structure (8 sections, 5 figures)

This paper contains 8 sections, 5 figures.

Table of Contents

  1. Introduction
  2. System Architecture & Hardware
  3. System Implementation
  4. Creating a 2D Map for Navigation
  5. Remote control via VR glasses
  6. Detection of the robot-server connectivity status
  7. Autonomous navigation
  8. Demo, Resources, and Future Developments

Figures (5)

  • Figure 1: The robotic platform highlighting the components and wireless connectivity. The red and blue lines represent the connection between each device, with dashed lines showing wireless connections and continuous lines showing wired connections.
  • Figure 2: The visualization of SLAM showing different areas: collision-free (light gray), unexplored (dark green), occupied and inaccessible areas (black).
  • Figure 3: Initial position of the robot on a) the physical test area and b) the virtual map.
  • Figure 4: The 3D virtual environment at the operator's end.
  • Figure 5: A snapshot at the $S_\alpha$ during the autonomous mode. Since there is no remote connectivity and control, the robot (black square) follows the calculated trajectory (black curve) towards the last known destination (red arrow).