Optimizing Robot Programming: Mixed Reality Gripper Control

TL;DR

This work analyzes controller-based mixed reality (MR) approaches to robot programming to address the inefficiencies of gesture-based methods. It compares Classical Jogging, Direct Control, and a Gripper-Control extension in a within-subjects study with $n=30$, evaluating task duration, user experience, workload, and performance. Results show Gripper Control consistently outperforms the other methods across most metrics, with Direct Control also improving over Classical Jogging, indicating a clear path toward more efficient MR robot programming. The findings suggest substantial practical impact for rapid, user-friendly programming, especially with potential hardware refinements and improved calibration techniques in future work.

Abstract

Conventional robot programming methods are complex and time-consuming for users. In recent years, alternative approaches such as mixed reality have been explored to address these challenges and optimize robot programming. While the findings of the mixed reality robot programming methods are convincing, most existing methods rely on gesture interaction for robot programming. Since controller-based interactions have proven to be more reliable, this paper examines three controller-based programming methods within a mixed reality scenario: 1) Classical Jogging, where the user positions the robot's end effector using the controller's thumbsticks, 2) Direct Control, where the controller's position and orientation directly corresponds to the end effector's, and 3) Gripper Control, where the controller is enhanced with a 3D-printed gripper attachment to grasp and release objects. A within-subjects study (n = 30) was conducted to compare these methods. The findings indicate that the Gripper Control condition outperforms the others in terms of task completion time, user experience, mental demand, and task performance, while also being the preferred method. Therefore, it demonstrates promising potential as an effective and efficient approach for future robot programming. Video available at https://youtu.be/83kWr8zUFIQ.

Figures (7)

• Figure 1: Illustration of the physical and virtual robot arm. The transparent shape represents the working envelope. The boxes on the left and right are used for the user study tasks.
• Figure 2: Construciton for synchronization of the virtual and physical robot bases. It is executed by inserting the 3D-printed gripper extension into the 3D-printed opening case.
• Figure 3: Visualization of the gripper used for condition Gripper Control.
• Figure 4: Target objects that were used for the two study tasks due to their self-correcting geometry.
• Figure 5: The means of the UEQ ratings for each condition. Scales are (PQ) Pragmatic Quality, (HQ) Hedonic Quality, and (O) Overall, ranging from -3 to +3. Error bars represent the standard error.