Investigating Size Congruency Between the Visual Perception of a VR Object and the Haptic Perception of Its Physical World Agent

TL;DR

The paper investigates size congruency between haptic perception of real objects and visual representations in VR, using two studies that combine real-world touch with VR size manipulation. A replacement size mechanism implemented in Unity maps fixed real-object sizes to a range of VR sizes and examines how tactile and visual cues influence size judgments, revealing stronger visual dominance for larger objects and a general tolerance for small-to-medium sizes. The work provides concrete thresholds and behavioral data, showing when users confirm or adjust sizes, and highlights practical implications for VR design in areas like sandplay therapy, surgical simulation, and virtual product manipulation. The findings offer a framework for aligning tactile feedback with visual feedback in VR and point to future work with larger samples and broader object shapes to enhance realism and immersion.

Abstract

The perception of physical objects and miniatures enhances the realism and immersion in VR. This work explores the relationship between haptic feedback from real objects and their visual representations in VR. The study examines how users confirm and adjust the sizes of different virtual objects. The results show that as the size of the virtual cubes increases, users are less likely to perceive the size correctly and need more adjustments. This research provides insights into how haptic sensations and visual inputs interact, contributing to the understanding of visual-haptic illusions in VR environments.

Figures (5)

• Figure 1: Experiment environment. (a) shows the experimental setup in VR. Participants face a white table with a cube on it. Under the guidance of the experimenter, participants need to touch the cube with a virtual hand and report whether the size matches the real cube they have touched. The other cubes in the figure represent the different sizes we will ask participants to touch, but only one cube will be displayed during the experiment. (b) shows the experimental setup in the real world. While participants touch the cube with a virtual hand, the experimenter will guide them to touch a real cube.
• Figure 2: Heat map of participant adjustment of size by participants. The X-axis represents the size of the cube touched by the participant in the real world (actual size), and the Y-axis represents the scale of the virtual cube size (virtual size) relative to the actual size in the VR environment (ranging from 60% to 140%). The color intensity of the heatmap indicates the proportion of adjustment made by the participant relative to the actual size when there is a perceived discrepancy between the actual and virtual sizes. Darker colors indicate greater adjustments. When the virtual size closely matches the actual size, the proportion of participant adjustments is smaller, suggesting that real tactile feedback can guide virtual perception to a certain extent. However, different actual sizes do not seem to affect this perception.
• Figure 3: Probability of not breaking the illusion by size category. The X-axis represents the actual size of the object touched by the participant, while the Y-axis represents the probability that participants do not perceive a difference between the virtual size and the actual size (i.e., the illusion is not broken) under different virtual sizes.
• Figure 4: Adjustment value by actual size. The X-axis represents the actual size of the object touched by the participant, while the Y-axis represents the adjustment value.
• Figure 5: Adjustment value by perceived size in VR. The X-axis represents the size of the virtual object touched by the participant, while the Y-axis represents the adjustment value.