Light Field Display Point Rendering

TL;DR

This work tackles the computational challenge of real-time light field display rendering by introducing Light Field Display Point Rendering (LFDPR), a point-based pipeline that extends prior eye-based rendering with texture-based splatting, LFD-biased sampling, and multiview mipmapping. By aligning triangle sampling to the LFD’s angular-spatial sampling and leveraging per-texture mipmaps across multiple views, LFDPR reduces the point cloud size and shader workload while mitigating aliasing and crosstalk through angular reconstruction and reconstruction filtering. The results demonstrate 2–8× speedups over conventional multiview rendering across multiple scenes with comparable image quality, and up to 3–5× gains under 2× spatial or angular supersampling, albeit with some speed penalties under certain configurations. These advances enable substantially faster real-time LFD rendering on commodity GPUs and provide methods (texture-based sampling, multiview mipmapping) that may also benefit other multiview rendering pipelines.

Abstract

Rendering for light field displays (LFDs) requires rendering of dozens or hundreds of views, which must then be combined into a single image on the display, making real-time LFD rendering extremely difficult. We introduce light field display point rendering (LFDPR), which meets these challenges by improving eye-based point rendering [Gavane and Watson 2023] with texture-based splatting, which avoids oversampling of triangles mapped to only a few texels; and with LFD-biased sampling, which adjusts horizontal and vertical triangle sampling to match the sampling of the LFD itself. To improve image quality, we introduce multiview mipmapping, which reduces texture aliasing even though compute shaders do not support hardware mipmapping. We also introduce angular supersampling and reconstruction to combat LFD view aliasing and crosstalk. The resulting LFDPR is 2-8x times faster than multiview rendering, with similar comparable quality.

Figures (10)

• Figure 1: Viewpoints distributed across an LFD horizontally see different pixels and views through the lenticular lens. Each view corresponds to one panel pixel under each lenslet, creating a tradeoff between angular and spatial resolution.
• Figure 2: Conventional image formation process for light field displays. A 3D scene is processed to generate a fronto-parallel light field which is then processed to be compatible with the lfd. When viewed from the lfd, the displayed image is perceived as a 3D scene fink2023_i3dLFD.
• Figure 3: LFDPR pipeline as implemented in the GPU. Improvements of iVIR and EPR are marked in red and green font respectively, and LFDPR innovations are marked in blue font.
• Figure 4: Reverse projection of pixel edges around the closest point on the polygon from the eye. The horizontal and vertical pixel edges often differ in length on LFDs.
• Figure 5: The proportion of an entire texture covered by pixels (LOD maps), as generated by lfdpr and mvr. lfdpr's multiview mipmapping generates two lod values for each texture in each triangle: (a) the minimum lod over all views, and (b) the maximum lod over all views. In (c), a particular view's LOD map is produced by interpolating between the minimum and maximum lod values. (d) presents the corresponding lod map in mvr.