Learned Display Radiance Fields with Lensless Cameras

TL;DR

This work tackles the challenge of calibrating displays across viewing directions without specialized hardware. It introduces a co-designed lensless camera and an Implicit Neural Representation (INR) to recover display light fields from lensless measurements, achieving ~46.6°×37.6° angular coverage with only nine training positions. The contributions include a lensless camera prototype with a phase mask and aperture array, a forward-modeling formulation that treats the diffuser as a near-field convolution, and an INR-based reconstruction trained end-to-end; results show ISO-aligned angular intensity trends and fast per-frame inference. The approach reduces the calibration burden and moves toward accessible, view-dependent display characterization, while acknowledging current limitations in full-panel coverage and scalability and suggesting future directions like end-to-end architectures and more efficient light-field representations.

Abstract

Calibrating displays is a basic and regular task that content creators must perform to maintain optimal visual experience, yet it remains a troublesome issue. Measuring display characteristics from different viewpoints often requires specialized equipment and a dark room, making it inaccessible to most users. To avoid specialized hardware requirements in display calibrations, our work co-designs a lensless camera and an Implicit Neural Representation based algorithm for capturing display characteristics from various viewpoints. More specifically, our pipeline enables efficient reconstruction of light fields emitted from a display from a viewing cone of 46.6° X 37.6°. Our emerging pipeline paves the initial steps towards effortless display calibration and characterization.

Figures (5)

• Figure 1: The display pixel captured with a microscope (left). The captured with the proposed lensless camera (middle). The captured lensless image from the display pixel (right).
• Figure 2: Each layer consists of 32 neurons and a sinusoidal activation function. We apply positional encoding with varying frequency levels ($L_f$) to each input coordinate group. After concatenating these encoded features, the model processes them to reconstruct the light field. We then perform linear convolution between this reconstruction and the pre-captured to generate the predicted lensless image ($\mathbf{I}\text{pred}$). Finally, we compute the loss with $\mathbf{I}\text{pred}$ and $\mathbf{I}_\text{gt}$.
• Figure 3: The incident angle ($\alpha$) of the light rays is determined by the distance between the aperture and the display pixel ($m$), as well as the size of the aperture ($n$) (left). Our proposed aperture array expands the incident angle range by turning on each pixel one by one (right).
• Figure 4: The top view of our lensless camera (left). The simulated display pixels with unseen incident angles and the corresponding overlay intensity heatmap (right).
• Figure 5: The illustration of the camera poses and the display (left). The average intensity that normalized to $[0,1]$ from the ISO standard and ours, with data samples color-coded by the corresponding camera poses (right).