Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VR$^2$: A Co-Located Dual-Headset Platform for Touch-Enabled Human-Robot Interaction Research

Chao Wang, Anna Belardinelli, Michael Gienger

TL;DR

This paper addresses the challenge of studying touch-rich human-robot interaction (HRI) by introducing VR2VR, a co-located dual-headset platform that lets a participant immersed in VR interact with a virtual robot while a second operator drives the robot in real space. The system leverages a shared spatial anchor to enable physically aligned touch and real-time retargeting of the operator's motion and facial signals to the robot, with flexible control over nonverbal channels. Key contributions include the system design, calibration workflow, safety considerations, and a Wizard-of-Oz demonstration that shows how VR2VR can rapidly prototype and rigorously evaluate embodied, touch-centric robot behaviors in a safer, controllable setting. The platform enables detailed multimodal data collection and facilitates iterative design and evaluation of social and tactile HRI in near-realistic scenarios, lowering barriers to prototyping and testing.

Abstract

Touch-rich human-robot interaction (HRI) is difficult to study: building and programming physical robots is costly and slow, while VR-based robot prototypes often remove physical contact or break the tight coupling between an agent's body and the user's felt touch. We present VR2VR, a co-located dual VR-headset platform for HRI research in which a participant and a hidden operator share the same physical space while experiencing different virtual embodiments. The participant sees an expressive virtual robot that interacts face-to-face in a shared virtual environment. In real time, the robot's upper-body gestures, head and gaze behaviors, and facial expressions are mapped from the operator's tracked motion and face signals. Because the operator is physically co-present and calibrated into the same coordinate frame, the operator can also physically touch the participant, enabling the participant to perceive robot touch aligned with the robot's hands; finger and hand motion are mapped to the robot using inverse kinematics to support precise contact. Beyond faithful motion retargeting for limb teleoperation, our VR2VR system supports experimental control by retargeting or selectively enabling nonverbal channels (e.g., head only vs. head+eyes vs. head+eyes+facial expressions) while keeping physical interaction constant. We detail the system design, calibration workflow, and safety considerations, and demonstrate the platform through a touch-based Wizard-of-Oz HRI study, illustrating how VR2VR lowers barriers for rapidly prototyping and rigorously evaluating embodied, touch-centric robot behaviors.

VR$^2$: A Co-Located Dual-Headset Platform for Touch-Enabled Human-Robot Interaction Research

TL;DR

This paper addresses the challenge of studying touch-rich human-robot interaction (HRI) by introducing VR2VR, a co-located dual-headset platform that lets a participant immersed in VR interact with a virtual robot while a second operator drives the robot in real space. The system leverages a shared spatial anchor to enable physically aligned touch and real-time retargeting of the operator's motion and facial signals to the robot, with flexible control over nonverbal channels. Key contributions include the system design, calibration workflow, safety considerations, and a Wizard-of-Oz demonstration that shows how VR2VR can rapidly prototype and rigorously evaluate embodied, touch-centric robot behaviors in a safer, controllable setting. The platform enables detailed multimodal data collection and facilitates iterative design and evaluation of social and tactile HRI in near-realistic scenarios, lowering barriers to prototyping and testing.

Abstract

Touch-rich human-robot interaction (HRI) is difficult to study: building and programming physical robots is costly and slow, while VR-based robot prototypes often remove physical contact or break the tight coupling between an agent's body and the user's felt touch. We present VR2VR, a co-located dual VR-headset platform for HRI research in which a participant and a hidden operator share the same physical space while experiencing different virtual embodiments. The participant sees an expressive virtual robot that interacts face-to-face in a shared virtual environment. In real time, the robot's upper-body gestures, head and gaze behaviors, and facial expressions are mapped from the operator's tracked motion and face signals. Because the operator is physically co-present and calibrated into the same coordinate frame, the operator can also physically touch the participant, enabling the participant to perceive robot touch aligned with the robot's hands; finger and hand motion are mapped to the robot using inverse kinematics to support precise contact. Beyond faithful motion retargeting for limb teleoperation, our VR2VR system supports experimental control by retargeting or selectively enabling nonverbal channels (e.g., head only vs. head+eyes vs. head+eyes+facial expressions) while keeping physical interaction constant. We detail the system design, calibration workflow, and safety considerations, and demonstrate the platform through a touch-based Wizard-of-Oz HRI study, illustrating how VR2VR lowers barriers for rapidly prototyping and rigorously evaluating embodied, touch-centric robot behaviors.
Paper Structure (11 sections, 1 equation, 3 figures)

This paper contains 11 sections, 1 equation, 3 figures.

Table of Contents

  1. Introduction
  2. Dual VR co-location system
  3. System features
  4. Co-location and asymmetric views
  5. Communication, logging, and touch aids
  6. Mapping human data to the robot
  7. Retargeting pipeline
  8. Conducting user studies
  9. Exemplary experimental design
  10. Data Collection
  11. Facial-expression mapping equations

Figures (3)

  • Figure 1: Human-to-robot mapping in VR2. (1) Face signals (selected blendshapes and gaze) control robot ear rotation and screen-eye shape/pose. (2) Head orientation is retargeted to the robot head; human palm pose provides end-effector targets and the robot arm joints are solved via IK. (3) Thumb and index fingertips follow the human fingertip poses via fingertip IK; remaining finger joints use direct joint-angle mapping.
  • Figure 2: Left: plot of synced gaze and touch events and FAU time series. Center, right: Instantaneous Face Action Units activations visualized as face expression and as bars for selected AUs.
  • Figure 3: The behavior of the virtual robot from the participant perspective during the experiment.