One Body, Two Minds: Alternating VR Perspective During Remote Teleoperation of Supernumerary Limbs

TL;DR

This work investigates dynamic guest-driven perspective switching in remote VR teleoperation with virtual supernumerary limbs (VSLs). It identifies limitations of fixed shared first-person viewpoints and introduces Embedded Anchored View and Out-of-body View to enable task-contingent switching. In a within-subjects study with 24 pairs (N=48), Out-of-body View improved navigation speed and reduced errors in manipulation tasks, while Embedded Anchored View supported near-body embodiment; results also reveal a trade-off between embodiment and spatial awareness, with switching costs and mode preferences shaped by task phase and user role. The work provides design guidelines for adaptive perspective switching in collaborative teleoperation and digital twins, informing future VR and robotics systems that combine multiple limbs and shared control.

Abstract

Remote VR teleoperation with supernumerary robotic limbs enables distant users to operate in another's local space. While a shared first-person view aids hand-eye coordination, locking the guest's camera to the host's head can degrade comfort, embodiment, and coordination. Based on a formative study (N=10) using a virtual supernumerary robotic limbs configuration to stress-test coordination, we propose guest-driven perspective switching from a shared first-person baseline (Shared Embodied View) to two alternatives: (a) a stabilized view with guest-controlled rotation (Embedded Anchored View), and (b) a fully decoupled third-person view (Out-of-body View). We ran a user study with 24 pairs (N=48) who switched between the baseline and proposed views as task demands changed. We measured performance, embodiment, fatigue, physiological arousal, and switching behaviors. Our results reveal role-dependent trade-offs: Out-of-body View improves navigation efficiency and reduces errors, while Embedded Anchored View supports embodiment. We conclude with guidelines: use Embedded Anchored View for hand-centric adjustments, Out-of-body View for navigation and object placement, and ensure smooth transitions.

Figures (8)

• Figure 1: Formative study configuration showing (a) the stable first-person perspective from the host, (b) the guest’s co-located but independently rotatable view illustrating visual instability, and (c) the avatar embodiment with VSLs used for near-body manipulation.
• Figure 2: Illustration of the Embedded Anchored View and Out-of-body View. (a) The guest user's viewpoint is spatially aligned with the host's viewpoint but rendered as a video-like window. (b) The guest user independently controls a virtual drone camera with 6 degrees-of-freedom (6-DoF), enabling flexible positional and rotational adjustments independent of the host's movements (the drone serves as a visual metaphor for the decoupled virtual camera).
• Figure 3: Guest perspectives organized as an Anchor–View matrix relative to the host. (a) Locked Egocentric: same anchor, same view; guest camera is rigidly locked to the host’s head pose and view. (Shown for comparison; not studied, as it grants the guest no agency and renders them a passive observer under full host control). (b) Shared Embodied View: same anchor, different view; guest shares the host’s position but rotates the camera independently. (c) Embedded Anchored View: different anchor, same view; guest remains independently positioned but views a stabilized window of the host’s egocentric feed. (d) Out-of-body View: different anchor, different view; guest drives a free-floating 6-DoF drone camera, decoupled from the host.
• Figure 4: Task overview: Transportation and Factory. Panels (a–b) depict the Transportation task: (a) colour-coded boxes placed on a central table; (b) matching colour-coded bins distributed throughout the VR environment. Panels (c–d) depict the Factory task: (c) geometrically distinct holes on a wall panel as precise insertion targets; (d) collaborative insertion of objects (e.g., cubes) into the corresponding holes.
• Figure 5: Physiological responses for hosts and guests under Embedded Anchored View and Out-of-body View across two different tasks. (a) HRV (RMSSD), a time–domain index of heart-rate variability. (b) Peak heart rate (bpm) by role and task. (c) Event-triggered heart rate aligned to perspective switches from the Shared Embodied View to the target view (Shared Embodied View$\rightarrow$Embedded Anchored View or Shared Embodied View$\rightarrow$Out-of-body View). * indicates p < .05 and ** indicates p < .01.