Breaking Down the Barriers: Investigating Non-Expert User Experiences in Robotic Teleoperation in UK and Japan

TL;DR

This work advances direct teleoperation by introducing TELESIM, a modular digital-twin framework that enables plug-and-play control of multiple robotic arms via 3D-pose controllers. It conducts a large, cross-cultural study (UK and Japan) using three robots (Baxter, UR3, UR5e) and two control interfaces to assess task performance, workload via NASA-TLX, and trust via NARS among non-expert users. The results show UR5e delivering the best tower-building performance with the lowest cognitive load, while hardware–control combinations can increase physical strain and frustration, and cultural factors influence trust. The findings highlight the importance of hardware capabilities and interface design for safe, usable teleoperation and provide a foundation for future exploration of additional control modalities and broader participant demographics.

Abstract

Robots are being created each year with the goal of integrating them into our daily lives. As such, there is an interest in research in evaluating the trust of humans toward robots. In addition, teleoperating robotic arms can be challenging for non-experts. To reduce the strain put on the user, we created TELESIM, a modular and plug-and-play framework that enables direct teleoperation of any robotic arm using a digital twin as the interface between users and the robotic system. We evaluated our framework using a user survey with three robots and control methods and recorded the user's workload and performance at completing a tower stacking task. However, an analysis of the strain on the user and their ability to trust robots was omitted. This paper addresses these omissions by presenting the additional results of our user survey of 37 participants carried out in United Kingdom. In addition, we present the results of an additional user survey, under similar conditions performed in Japan, with the goal of addressing the limitations of our previous approach, by interfacing a VR controller with a UR5e. Our experimental results show that the UR5e has more towers built. Additionally, the UR5e gives the least amount of cognitive stress, while the combination of Senseglove and UR3 provides the user with the highest physical strain and causes the user to feel more frustrated. Finally, the Japanese participants seem more trusting of robots than the British participants.

Figures (12)

• Figure 1: Our framework TELESIM is being used to control a UR3 Robot (top-right, top-center) and a Baxter Robot (top-left) in the United Kingdom and a UR5e robot in Japan (bottom)
• Figure 2: Overview of TELESIM. The controllers (in the black dotted line) (1) can be any system that outputs a 3D pose. TELESIM is depicted in the blue dotted line, which accepts the pose given by (1) to update the 3D pose of a cube in the digital twin. The robot then calculates a path to this cube in real-time while avoiding collision with the world (4). Finally, as shown in the red dotted line, TELESIM can be plugged into any robotic system (6) via a ROS2 robot controller (5).
• Figure 3: Photo of the modified T42 Gripper with the second level added for the control boards
• Figure 4: Overview of the experimental setup. The Steam Index VR Headset noauthor_valve_nodate is marked as (1) on the far left, which acts as the world's origin. The Baxter robot on the left (2) is controlled by the Steam Index controller (5). In front of it, the UR3 is on the right (3), with the Yale OpenHand T42 gripper noauthor_yale_2023, controlled by the Senseglove and HTC Vive tracker (4) on the left side of the brown table. Additionally, in the upper right corner (7), a view of the starting setup of the task, which consists of 3 cubes in a triangle pattern (described in Section \ref{['sec:experimental_setup']}), while on the brown table, the cubes are arranged in the goal configuration (6).
• Figure 5: Overview of our experimental setup in AIST. The Steam VR headset (1) acts as the world's origin, as seen behind the robot and the user. The UR5e robot (2) in front is controlled by a Steam Index VR controller (3). There are 3 cubes set up in their starting position (4) in a triangular pattern similar to audonnet_telesim_2024. The empty square in the middle of the table (5) represents the location where the user should stack all the cubes, resulting in a tower of 3 cubes.