Should Teleoperation Be like Driving in a Car? Comparison of Teleoperation HMIs

TL;DR

This paper investigates how teleoperation HMIs can be optimized by separating path and velocity input, comparing three prototypes against Direct Control using a miniature vehicle on a short track. A ROS2-based tod stack implementation enables path via waypoints and velocity control through distinct inputs, with performance evaluated by expert users on task time, usability, and workload. Results show parallel steering-wheel control is fastest but most demanding, while sequential path/velocity input (mouse/keyboard or touchscreen) reduces workload and is preferred in non-time-critical contexts, though at the cost of speed. The findings yield design guidelines for unified displays, intuitive velocity controls, and collision-awareness overlays, and point to future work in integrating waypoint-based guidance and testing in real-world vehicle scenarios.

Abstract

Since Automated Driving Systems are not expected to operate flawlessly, Automated Vehicles will require human assistance in certain situations. For this reason, teleoperation offers the opportunity for a human to be remotely connected to the vehicle and assist it. The Remote Operator can provide extensive support by directly controlling the vehicle, eliminating the need for Automated Driving functions. However, due to the physical disconnection to the vehicle, monitoring and controlling is challenging compared to driving in the vehicle. Therefore, this work follows the approach of simplifying the task for the Remote Operator by separating the path and velocity input. In a study using a miniature vehicle, different operator-vehicle interactions and input devices were compared based on collisions, task completion time, usability and workload. The evaluation revealed significant differences between the three implemented prototypes using a steering wheel, mouse and keyboard or a touchscreen. The separate input of path and velocity via mouse and keyboard or touchscreen is preferred but is slower compared to parallel input via steering wheel.

Figures (7)

• Figure 1: A teleoperation system comprises a teleoperation concept, a user interface and a safety concept
• Figure 2: Three implemented hmi variants with the display of camera image and map on separate monitors: Parallel path and velocity input using a steering wheel, Path input via mouse clicks on map and velocity input using keys, Path input via stylus clicks on touchscreen map and velocity input using physical touchscreen buttons
• Figure 3: The set waypoints are visualized as green squares and connected by a center line with two parallel lines marking the vehicle's width. The red center line indicates impassable curve radii, with the red path simulating the actual vehicle behavior.
• Figure 4: An overview of implemented modules and interfaces of the teleoperation system for separate path and velocity input. Arrows mark communication interfaces.
• Figure 5: The boxplots show that the parallel input of path and velocity via steering wheel (Mdn = $80s$) was significantly faster than the separate input via mouse and keyboard (Mdn = $101s$) or touchscreen (Mdn = $111s$).