Guided Lensless Polarization Imaging

Abstract

Polarization imaging captures the polarization state of light, revealing information invisible to the human eye yet valuable in domains such as biomedical diagnostics, autonomous driving, and remote sensing. However, conventional polarization cameras are often expensive, bulky, or both, limiting their practical use. Lensless imaging offers a compact, low-cost alternative by replacing the lens with a simple optical element like a diffuser and performing computational reconstruction, but existing lensless polarization systems suffer from limited reconstruction quality. To overcome these limitations, we introduce a RGB-guided lensless polarization imaging system that combines a compact polarization-RGB sensor with an auxiliary, widely available conventional RGB camera providing structural guidance. We reconstruct multi-angle polarization images for each RGB color channel through a two-stage pipeline: a physics-based inversion recovers an initial polarization image, followed by a Transformer-based fusion network that refines this reconstruction using the RGB guidance image from the conventional RGB camera. Our two-stage method significantly improves reconstruction quality and fidelity over lensless-only baselines, generalizes across datasets and imaging conditions, and achieves high-quality real-world results on our physical prototype lensless camera without any fine-tuning.

Figures (10)

• Figure 1: RGB-guided lensless polarization imaging system: (a) optical setup; (b) custom polarization mask; (c) captured lensless image under front illumination with two orthogonally polarized projectors; and (d) reconstructed grayscale polarization result, visualized by mapping the $0^\circ$, $45^\circ$, and $90^\circ$ outputs to the R, G, and B channels.
• Figure 1: Illustration of our polarization mask, containing four repetitions of striped polarizers at orientations $0^\circ, 45^\circ, 90^\circ,\text{and } 135^\circ$
• Figure 2: Overview of the proposed RGB-guided reconstruction pipeline. The process consists of two stages: (1) polarization intensity images (color or grayscale) are reconstructed from lensless measurements using a physics-based algorithm (FISTA/ADMM); and (2) the initial reconstruction and a registered RGB image of the same scene are separately encoded and fused through cross-domain attention to produce a refined polarization reconstruction. For visualization, the grayscale reconstructions at three polarization angles ($0^\circ$, $45^\circ$, $90^\circ$) are mapped to the R, G, and B channels. The pipeline is compatible with more general input configurations.
• Figure 2: The experimental optical setup from two viewpoints. Left: Front view showing the imaging target and the two-sensor setup: (1) lensless polarization camera and (2) RGB reference camera. Right: Back view of the setup showing (3) the rotation stage with a mounted linear polarizer positioned in front of (2) the RGB camera for capturing reference images. The lensless camera prototype (1) consists of a diffuser and a manually assembled polarization mask mounted on the sensor.
• Figure 3: Real and simulated polarization mask responses for four polarization angles in grayscale. (Top) Real masks captured with our prototype; (center) simulated masks; and (bottom) measured grayscale PSF of our diffuser in spatial and frequency domains. The PSF is low-pass, leading to loss of high-frequency information.