Visual Acuity Consistent Foveated Rendering towards Retinal Resolution

TL;DR

We address the challenge of retinal-resolution VR rendering by deriving a visual acuity-based log-polar mapping that yields a constant, resolution-independent LP buffer and a shading-rate distribution aligned with human acuity. VaFR eliminates dependence on device-specific parameters, providing stable performance across display resolutions and gaze directions, and supports rasterization, ray casting, and path tracing pipelines. The approach achieves substantial speedups (e.g., $SR$-driven gains up to $16.4\times$ in ray casting at retinal resolution) while preserving perceptual quality, as demonstrated by quantitative metrics and user studies. This work advances ultra-high-definition VR rendering by enabling retinal-detail foveation that scales with future HMD resolutions and maintains interactive frame rates for immersive experiences.

Abstract

Prior foveated rendering methods often suffer from a limitation where the shading load escalates with increasing display resolution, leading to decreased efficiency, particularly when dealing with retinal-level resolutions. To tackle this challenge, we begin with the essence of the human visual system (HVS) perception and present visual acuity-consistent foveated rendering (VaFR), aiming to achieve exceptional rendering performance at retinal-level resolutions. Specifically, we propose a method with a novel log-polar mapping function derived from the human visual acuity model, which accommodates the natural bandwidth of the visual system. This mapping function and its associated shading rate guarantee a consistent output of rendering information, regardless of variations in the display resolution of the VR HMD. Consequently, our VaFR outperforms alternative methods, improving rendering speed while preserving perceptual visual quality, particularly when operating at retinal resolutions. We validate our approach using both the rasterization and ray-casting rendering pipelines. We also validate our approach using different binocular rendering strategies for HMD devices. In diverse testing scenarios, our approach delivers better perceptual visual quality than prior foveated rendering while achieving an impressive speedup of 6.5$\times$-9.29$\times$ for deferred rendering of 3D scenarios and an even more powerful speedup of 10.4$\times$-16.4$\times$ for ray-casting at retinal resolution. Additionally, our approach significantly enhances the rendering performance of binocular 8K path tracing, achieving smooth frame rates.

Figures (21)

• Figure 1: Results of our visual acuity consistent foveated rendering (VaFR) in the middle, ground truth (GT) on the left, and locomotion-aware foveated rendering (LaFR Shi:2023) on the right, rendered in ray casting mode at 8K ($7680\times4320$) resolution. Our approach, achieving an extraordinarily high frame rate of 210 fps for binocular rendering in VR settings, delivers comparable perceptual quality to that of LaFR (49 fps) and GT (23 fps).
• Figure 2: Schematic diagram of log-polar mapping and its inverse between the screen space and the log-polar space associated with the LP buffer, along with key variables in \ref{['sec:mapping function']}. Additionally, we highlight the variable shading height $l(e)$, which differs from the buffer height $h$
• Figure 3: Our pivotal points for segmented mapping function, Standard shading rate, two curves adapted to VIVE EYE and Pimax Crystal.
• Figure 4: Working pipeline of our visual acuity consistent foveated rendering in the deferred rendering pipeline. The LP buffer dimensions are 932 $\times$ 1800, yet it includes only 1,061,735 shading points.
• Figure 5: Foveated 3D graphics Guenter:2012 uses a piecewise step function, causing shading rate discontinuities. LMFR Meng:2018:Kernel introduces a continuous shading rate but still fails to align with human visual acuity. LaFR identifies LMFR's suboptimal quality and refines it by segmenting the kernel function and fine-tuning parameters to better adapt shading distribution. RMFR Ye:2022 plots all points along the x-axis ($f_x=0.2W, f_y=0.2H, \delta=2.6$, gaze-centered, as per the original paper). Similarly, Visual-Polar koskela2019foveated exhibits a shading rate far below visual acuity in the foveal region.