The perceptual gap between video see-through displays and natural human vision

TL;DR

This work addresses the perceptual fidelity gap between video see-through head-mounted displays (VST HMDs) and natural human vision. It introduces a low-cost, end-to-end visual perception benchmark implemented in Unity to quantify acuity, contrast sensitivity, and color vision for three devices (Vision Pro, Quest 3, Quest Pro) under normal and low lighting, compared against naked-eye performance. Key findings show that all tested VST HMDs fail to match natural vision, with pronounced degradation in low-light conditions; Vision Pro generally offers the strongest performance among the devices, while Quest Pro lags across metrics. The study highlights the need for standardized perceptual metrics, transparency in hardware parameters, and compensation strategies to advance VST fidelity for real-world use cases.

Abstract

Video see-through (VST) technology aims to seamlessly blend virtual and physical worlds by reconstructing reality through cameras. While manufacturers promise perceptual fidelity, it remains unclear how close these systems are to replicating natural human vision across varying environmental conditions. In this work, we quantify the perceptual gap between the human eye and different popular VST headsets (Apple Vision Pro, Meta Quest 3, Quest Pro) using psychophysical measures of visual acuity, contrast sensitivity, and color vision. We show that despite hardware advancements, all tested VST systems fail to match the dynamic range and adaptability of the naked eye. While high-end devices approach human performance in ideal lighting, they exhibit significant degradation in low-light conditions, particularly in contrast sensitivity and acuity. Our results map the physiological limitations of digital reality reconstruction, establishing a specific perceptual gap that defines the roadmap for achieving indistinguishable VST experiences.

Figures (9)

• Figure 1: Screenshots of the visual acuity ((a) and (b)) and contrast sensitivity tests ((c) and (d)). The first and second rows ((a) (c), (b) (d)) are start and end state examples of the two tests.
• Figure 2: Screenshots of the digital version of the Farnsworth-Munsell 100 Hue test. The first and second rows are start and end state examples of the 100-Hue test.
• Figure 3: Pictures of the testing environment. (a) and (c): a 100-Hue test using the monitor. (b) and (d): tests using a smartphone. (a) and (b) were taken under the normal-light level (572 lux). (c) and (d) were taken under the low-light level (117 lux).
• Figure 4: The polynomial fit of grayscale vs Weber contrast of Google Pixel 3 XL under 100% screen brightness.
• Figure 5: Violin plots with post-hoc results of the visual perception benchmark dataset. '*' to '***' represent Bonferroni-adjusted significant differences at '.05', '.01', '.001' level. The green area represents normal and above normal logMAR and logCS values, superior and above superior values for TES.