Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Robotic Haptic Proxies in Virtual Reality

Eric Godden, Matthew Pan

TL;DR

The paper addresses enabling tactile interaction in VR by developing a robotic haptic proxy that co-locates a physical hand with a virtual hand. It presents a system integrating a KUKA LBR iiwa arm with an inside-out tracked Meta Quest Pro, using IK and a real-time motion planner to teleoperate the robot from VR hand poses while streaming data over UDP/Protocol Buffers. The study provides quantitative latency and accuracy measurements and demonstrates a teleoperation task as a proof of concept. The findings highlight latency and positional error as key bottlenecks and set the stage for future latency compensation and higher-fidelity haptic rendering to improve VR presence.

Abstract

This work represents the initial development of a haptic display system for increased presence in virtual experiences. The developed system creates a two-way connection between a virtual space, mediated through a virtual reality headset, and a physical space, mediated through a robotic manipulator, creating the foundation for future haptic display development using the haptic proxy framework. Here, we assesses hand-tracking performance of the Meta Quest Pro headset, examining hand tracking latency and static positional error to characterize performance of our system.

Towards Robotic Haptic Proxies in Virtual Reality

TL;DR

The paper addresses enabling tactile interaction in VR by developing a robotic haptic proxy that co-locates a physical hand with a virtual hand. It presents a system integrating a KUKA LBR iiwa arm with an inside-out tracked Meta Quest Pro, using IK and a real-time motion planner to teleoperate the robot from VR hand poses while streaming data over UDP/Protocol Buffers. The study provides quantitative latency and accuracy measurements and demonstrates a teleoperation task as a proof of concept. The findings highlight latency and positional error as key bottlenecks and set the stage for future latency compensation and higher-fidelity haptic rendering to improve VR presence.

Abstract

This work represents the initial development of a haptic display system for increased presence in virtual experiences. The developed system creates a two-way connection between a virtual space, mediated through a virtual reality headset, and a physical space, mediated through a robotic manipulator, creating the foundation for future haptic display development using the haptic proxy framework. Here, we assesses hand-tracking performance of the Meta Quest Pro headset, examining hand tracking latency and static positional error to characterize performance of our system.
Paper Structure (8 sections, 2 figures)

This paper contains 8 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Background
  3. System
  4. Results and Discussion
  5. Latency
  6. Position Accuracy
  7. Teleoperation Task
  8. Conclusions and Future Work

Figures (2)

  • Figure 1: System setup to measure hand tracking latency of the Meta Quest Pro headset. The robot moves the replica hand in a sinusoid motion while hand pose data, as seen by the headset, is recorded. Phase differences between the robot- and headset-reported motion are used to determine latency.
  • Figure 2: Teleoperation task illustrating aligned user-robot motion.