Neural Étendue Expander for Ultra-Wide-Angle High-Fidelity Holographic Display

TL;DR

The paper tackles the limited étendue of holographic displays, which constrains either the field of view or display size under practical SLM resolutions. It introduces neural étendue expanders learned from natural images via an end-to-end differentiable holographic model, jointly optimizing the static expander and dynamic SLM patterns to maximize fidelity while expanding diffraction angles. In simulations and experiments, the approach achieves up to $64\times$ étendue expansion with >$29\,\mathrm{dB}$ PSNR on retinal-resolution content, and prototypes built with a $1\mathrm{K}$-pixel SLM validate ultra-wide-field, high-fidelity color holography; scalable to $8\mathrm{K}$ SLM to cover about $85\%$ of the human stereo FOV with an $18.5\mathrm{mm}$ eyebox. This work enables practical ultra-wide-angle holographic VR/AR displays and points toward future metasurface-based extensions to further enlarge diffraction angles. $I = |\mathcal{F}(\mathcal{E} \odot U(\mathcal{S}))|^2$ and retinal-frequency filtering underpin the design, ensuring reconstructed content aligns with human perceptual limits.

Abstract

Holographic displays can generate light fields by dynamically modulating the wavefront of a coherent beam of light using a spatial light modulator, promising rich virtual and augmented reality applications. However, the limited spatial resolution of existing dynamic spatial light modulators imposes a tight bound on the diffraction angle. As a result, modern holographic displays possess low étendue, which is the product of the display area and the maximum solid angle of diffracted light. The low étendue forces a sacrifice of either the field-of-view (FOV) or the display size. In this work, we lift this limitation by presenting neural étendue expanders. This new breed of optical elements, which is learned from a natural image dataset, enables higher diffraction angles for ultra-wide FOV while maintaining both a compact form factor and the fidelity of displayed contents to human viewers. With neural étendue expanders, we experimentally achieve 64$\times$ étendue expansion of natural images in full color, expanding the FOV by an order of magnitude horizontally and vertically, with high-fidelity reconstruction quality (measured in PSNR) over 29 dB on retinal-resolution images.

Figures (3)

• Figure 1: Neural étendue expansion for ultra-wide angle, high-fidelity holograms.a Conventional holographic displays suffer from low étendue, resulting in either small FOV or eyebox size. Here, we illustrate a small FOV as $\theta_s$. b Introducing a neural étendue expander into the display facilitates ultra-wide angle holograms, here we illustrate the increase in FOV as $\theta_n$. c We design the neural étendue expanders via an end-to-end optimization algorithm that considers the SLM wavefront modulation and the human viewer's perceptual response. One SLM pattern is optimized for each training sample, while the neural étendue expander learns a general structure that facilitates hologram generation of any natural image. d The learned neural étendue expander preserves the major frequency bands of natural images within the frequency cutoff determined by the resolution of the human retina.
• Figure 2: Experimental demonstration of neural étendue expansion.a Schematic of holographic display prototype with the neural étendue expander inserted at the conjugate plane of the SLM. b Microscope images of a fabricated neural étendue expander and a binary random expanderkuo2020expansion. c Captures of holograms generated with the display prototype. The small dark circle in the center of the pictures corresponds to the DC block. Left: Non-étendue expanded holograms produced with conventional holographyshi2021towards. These holograms have extremely low FOV. Middle: $64\times$ étendue expanded holograms produced with the binary random expander show low contrast and chromatic artifacts. Right: $64\times$ étendue expanded holograms produced with the neural étendue expander show high fidelity. d Decomposition of $64\times$ étendue expanded holograms into constituent colors ($450nm$, $517nm$, $660nm$). We observe improved hologram contrast and less scatter with neural étendue expansion, even at the wavelength of $660nm$ which was used to design the binary random expander.
• Figure 3: Étendue expander characterization.a The 64$\times$ étendue expanded holograms generated with neural étendue expanders have the highest fidelity with respect to the target natural image, for both the trichromatic and monochromatic cases. In comparison, the holograms generated with binary random expanderskuo2020expansion or photon sievesPark2019UltrathinWL show lower contrast and more speckle noise. Photon sieves could generate étendue expanded holograms of sparse points but not of natural scenes. A low étendue hologram generated with conventional holographyshi2021towards and no expander is included for comparison. b Quantitative performance comparison of $64\times$ étendue expanded holograms when using neural, uniform random, and binary random expanders. The metrics are evaluated over an unseen test set. c Virtual frequency modulation cross section. Neural étendue expanders push reconstruction artifacts outside of the perceivable frequency bands of human vision while producing a natural image frequency spectrum within the passband as predicted by Eq. \ref{['eq:proof2']}. In contrast, the both uniform and binary random expanders exhibit a flat spectrum which reduces the reconstruction quality within the passband. The cutoff frequency is indicated by $c$. d Quantitative reconstruction quality of neural étendue expansion, random expansion, and photon sievesPark2019UltrathinWL for increasing étendue expansion factors for the monochromatic case. Uniform and binary random expansion both achieve the same performance for a single wavelength. e Visualization of the learned expanders for increasing étendue expansion factors. We observe that the learned modulation structures contain both high and low frequency components. f Visualization of the corresponding virtual frequency modulation for each expander.