DirectL: Efficient Radiance Fields Rendering for 3D Light Field Displays

TL;DR

DirectL presents a principled, low-level fusion of Radiance Fields with lenticular light-field displays, addressing both content creation and rendering efficiency. By subpixel repurposing and tailored pipelines for NeRFs and 3D Gaussian Splatting, it achieves up to 40× speedups while maintaining perceptual quality, enabling real-time rendering on multiple display sizes. The approach reduces the encoded image workload and demonstrates strong potential for downstream radiance-field tasks and commercial deployment. This work broadens the accessibility of 3D content creation and offers a practical path toward immersive naked-eye 3D experiences on consumer and large-format displays.

Abstract

Autostereoscopic display, despite decades of development, has not achieved extensive application, primarily due to the daunting challenge of 3D content creation for non-specialists. The emergence of Radiance Field as an innovative 3D representation has markedly revolutionized the domains of 3D reconstruction and generation. This technology greatly simplifies 3D content creation for common users, broadening the applicability of Light Field Displays (LFDs). However, the combination of these two fields remains largely unexplored. The standard paradigm to create optimal content for parallax-based light field displays demands rendering at least 45 slightly shifted views preferably at high resolution per frame, a substantial hurdle for real-time rendering. We introduce DirectL, a novel rendering paradigm for Radiance Fields on 3D displays. We thoroughly analyze the interweaved mapping of spatial rays to screen subpixels, precisely determine the light rays entering the human eye, and propose subpixel repurposing to significantly reduce the pixel count required for rendering. Tailored for the two predominant radiance fields--Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS), we propose corresponding optimized rendering pipelines that directly render the light field images instead of multi-view images. Extensive experiments across various displays and user study demonstrate that DirectL accelerates rendering by up to 40 times compared to the standard paradigm without sacrificing visual quality. Its rendering process-only modification allows seamless integration into subsequent radiance field tasks. Finally, we integrate DirectL into diverse applications, showcasing the stunning visual experiences and the synergy between LFDs and Radiance Fields, which unveils tremendous potential for commercialization applications. \href{direct-l.github.io}{\textbf{Project Homepage}

Figures (7)

• Figure 1: The overall of standard lenticular lens based 3D display system.
• Figure 2: Overview of the DirectL. Initially, ray configurations and the requisite index for rearrangement are calculated offline based on the hardware parameters of any given lenticular 3D display. Subsequently, the origin $\bm o_i$ and direction $\bm d_i$ of each ray are determined according to ray configurations and the current center of the light field. Following this, view-independent ray-order rendering of the radiance field is conducted to ascertain the color of each ray. Finally, the G and B channels are reordered to produce a light field image ready for direct display.
• Figure 3: Our subpixel repurposing scheme. In the standard paradigm, the one-to-one mapping approach only leverages a single subpixel from the multi-viewpoint image pixels, with the remaining subpixels being discarded. Following the display principle, we repurpose these discarded subpixels during the interlacing process, thereby reducing the number of pixels that require rendering.
• Figure 4: Comparison of the ratio of pixels rendered by different methods to the native screen resolution. For per-pixel rendering radiance field methods such as NeRFs, the number of rendered pixels directly impacts efficiency.
• Figure 5: Comparison of display effects between DirectL and standard paradigm. LR suffers from severe blurring due to extensive upsampling during interweaving, which also leads to the swelling of object edges. MR alleviates these issues to some extent but still encounters them. DirectL offers display quality nearly identical to HR, and in some cases, even superior. Due to real photography, perfect alignment of corresponding image positions is not always achievable.