Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mixed Reality Outperforms Virtual Reality for Remote Error Resolution in Pick-and-Place Tasks

Advay Kumar, Stephanie Simangunsong, Pamela Carreno-Medrano, Akansel Cosgun

TL;DR

This paper addresses remote error resolution in robotic pick-and-place tasks by comparing three visualization interfaces—Mixed Reality (MR), Virtual Reality (VR), and camera streams—using the same headset and controller to control a robotic arm. Through a within-subject user study (N=21), MR consistently enabled faster task completion, higher usability, and lower perceived workload than VR and camera interfaces, with participants preferring MR overall. The findings suggest that MR’s blend of real-world context and virtual guidance improves spatial understanding and reduces cognitive load, highlighting its practical potential for industrial human-robot collaboration. Limitations include a simplified error scenario and a relatively small, homogeneous participant pool; future work should test more complex faults and involve a broader set of operators to validate MR’s advantages in real-world settings.

Abstract

This study evaluates the performance and usability of Mixed Reality (MR), Virtual Reality (VR), and camera stream interfaces for remote error resolution tasks, such as correcting warehouse packaging errors. Specifically, we consider a scenario where a robotic arm halts after detecting an error, requiring a remote operator to intervene and resolve it via pick-and-place actions. Twenty-one participants performed simulated pick-and-place tasks using each interface. A linear mixed model (LMM) analysis of task resolution time, usability scores (SUS), and mental workload scores (NASA-TLX) showed that the MR interface outperformed both VR and camera interfaces. MR enabled significantly faster task completion, was rated higher in usability, and was perceived to be less cognitively demanding. Notably, the MR interface, which projected a virtual robot onto a physical table, provided superior spatial understanding and physical reference cues. Post-study surveys further confirmed participants' preference for MR over other interfaces.

Mixed Reality Outperforms Virtual Reality for Remote Error Resolution in Pick-and-Place Tasks

TL;DR

This paper addresses remote error resolution in robotic pick-and-place tasks by comparing three visualization interfaces—Mixed Reality (MR), Virtual Reality (VR), and camera streams—using the same headset and controller to control a robotic arm. Through a within-subject user study (N=21), MR consistently enabled faster task completion, higher usability, and lower perceived workload than VR and camera interfaces, with participants preferring MR overall. The findings suggest that MR’s blend of real-world context and virtual guidance improves spatial understanding and reduces cognitive load, highlighting its practical potential for industrial human-robot collaboration. Limitations include a simplified error scenario and a relatively small, homogeneous participant pool; future work should test more complex faults and involve a broader set of operators to validate MR’s advantages in real-world settings.

Abstract

This study evaluates the performance and usability of Mixed Reality (MR), Virtual Reality (VR), and camera stream interfaces for remote error resolution tasks, such as correcting warehouse packaging errors. Specifically, we consider a scenario where a robotic arm halts after detecting an error, requiring a remote operator to intervene and resolve it via pick-and-place actions. Twenty-one participants performed simulated pick-and-place tasks using each interface. A linear mixed model (LMM) analysis of task resolution time, usability scores (SUS), and mental workload scores (NASA-TLX) showed that the MR interface outperformed both VR and camera interfaces. MR enabled significantly faster task completion, was rated higher in usability, and was perceived to be less cognitively demanding. Notably, the MR interface, which projected a virtual robot onto a physical table, provided superior spatial understanding and physical reference cues. Post-study surveys further confirmed participants' preference for MR over other interfaces.

Paper Structure

This paper contains 18 sections, 5 figures, 5 tables.

Table of Contents

  1. Introduction and Related Works
  2. Methodology
  3. Error Scenario
  4. Scenario Implementation
  5. Visualisation
  6. Virtual Reality (VR)
  7. Mixed Reality (MR)
  8. Camera Streams
  9. Error Correction User Study
  10. Study Hypotheses
  11. Study Setup and Procedure
  12. Metrics
  13. Objective Metrics
  14. Subjective Metrics
  15. Participants
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Scenario used in the study including the robot, cameras, bins and cubes
  • Figure 2: Visualisations presented to participants during the user study.
  • Figure 3: Distributions of usability scores for each interface
  • Figure 4: Distribution of Nasa TLX scores for each interface
  • Figure 5: Post-study survey result regarding participants preferred each interface and which interface they found to be the most comfortable.