Disentangling Coordiante Frames for Task Specific Motion Retargeting in Teleoperation using Shared Control and VR Controllers

TL;DR

This work tackles the gap between teleoperation performance and direct manipulation by introducing a formal motion retargeting framework that decouples translation and rotation through two independent coordinate-frame trees. Integrated with a model predictive control trajectory planner, the approach maps VR-controller inputs to robot commands in a way that supports both absolute and relative retargeting, reducing misalignment due to viewpoint changes. The method is validated on a UR5e manipulator with a VR controller, achieving real-time planning (solve times < 7 ms) and practical end-to-end latency around 0.5 s, using a horizon of $H=10$ steps and $w_{1p}=w_{1o}=[100,100,100]$, with $Q_{2a}=Q_{2b}$ diagonal entries of $0.01$. The results demonstrate that translation-rotation separation improves intuitive control and lays groundwork for low-cost, shared-control teleoperation in challenging environments, while noting the need for broader user studies and exploration of alternative reference-frame strategies.

Abstract

Task performance in terms of task completion time in teleoperation is still far behind compared to humans conducting tasks directly. One large identified impact on this is the human capability to perform transformations and alignments, which is directly influenced by the point of view and the motion retargeting strategy. In modern teleoperation systems, motion retargeting is usually implemented through a one time calibration or switching modes. Complex tasks, like concatenated screwing, might be difficult, because the operator has to align (e.g. mirror) rotational and translational input commands. Recent research has shown, that the separation of translation and rotation leads to increased task performance. This work proposes a formal motion retargeting method, which separates translational and rotational input commands. This method is then included in a optimal control based trajectory planner and shown to work on a UR5e manipulator.

Figures (4)

• Figure 1: Teleoperated robotic manipulator with input device {d}. As the end effector {e} is facing the viewpoint of the operator, the operator has to mirror his input commands for translation and rotation.
• Figure 2: Visualization of the two trees of coordinate frames. The tree of frames on the left (input tree) describes the input device with its origin frame $\{0_I\}$, and the tree on the right (robot tree) represents the robot manipulator with the tool center point frame $\{e\}$ and its origin frame $\{0_M\}$. The image of the VR controller was AI-generated.
• Figure 3: Depiction of a rotational movement command. With the clutching in the first image, the reference frame $\{r_I\}$ is set with the current orientation of the input device $\{d\}$. The orientation for the reference frames $\{t_I\}$ and $\{t_M\}$ was set during a calibration phase. Thus a movement of the VR-controller to the left (from the image perspective) will lead to a movement of the end effector to the left during all stages of this scene.
• Figure 4: Reference and measured orientation of the end effector and the VR-controller for roll and pitch angles. The bottom graph shows the activation of the clutch button by the operator.

Theorems & Definitions (3)

• Definition 1
• Definition 2
• Definition 3