Dispersed Structured Light for Hyperspectral 3D Imaging

TL;DR

This work presents Dispersed Structured Light (DSL), a cost-effective hyperspectral 3D imaging method that places a sub-millimeter diffraction grating film in front of a conventional projector to disperse structured light by wavelength. By developing a dispersive projection image formation model that accounts for zero- and first-order diffractions and a per-pixel hyperspectral reconstruction pipeline, the authors achieve a depth accuracy around $1\,\mathrm{mm}$ and a spectral FWHM of $18.8\,\mathrm{nm}$ in the visible, using a compact prototype costing about $10$ USD. Key contributions include a data-driven first-order correspondence model with $\approx 1$ pixel reprojection error, a two-stage depth+spectral reconstruction strategy using binary-code and scanline patterns, and thorough calibration and validation on real scenes, color targets, and metameric samples. DSL demonstrates that accurate hyperspectral 3D imaging can be made practical and accessible for domains such as culture heritage, geology, and biology, by preserving measurement quality while maintaining a small form factor and low cost.

Abstract

Hyperspectral 3D imaging aims to acquire both depth and spectral information of a scene. However, existing methods are either prohibitively expensive and bulky or compromise on spectral and depth accuracy. In this work, we present Dispersed Structured Light (DSL), a cost-effective and compact method for accurate hyperspectral 3D imaging. DSL modifies a traditional projector-camera system by placing a sub-millimeter thick diffraction grating film front of the projector. The grating disperses structured light based on light wavelength. To utilize the dispersed structured light, we devise a model for dispersive projection image formation and a per-pixel hyperspectral 3D reconstruction method. We validate DSL by instantiating a compact experimental prototype. DSL achieves spectral accuracy of 18.8nm full-width half-maximum (FWHM) and depth error of 1mm. We demonstrate that DSL outperforms prior work on practical hyperspectral 3D imaging. DSL promises accurate and practical hyperspectral 3D imaging for diverse application domains, including computer vision and graphics, cultural heritage, geology, and biology.

Figures (9)

• Figure 1: We introduce dispersed structured light, a low-cost high-quality hyperspectral 3D imaging method. By placing a diffraction grating film on a conventional camera-projector setup, we disperse structured-light patterns. Analyzing the images captured under the dispersed structured light enables accurate hyperspectral 3D reconstruction. (a) Capture configuration, (b) estimated hyperspectral image in sRGB, (c) comparison with spectroradiometer measurements, (d) estimated depth map, (e) estimated hyperspectral image.
• Figure 2: Experimental prototype. (a) & (b) Our prototype consists of an RGB projector equipped with a diffraction grating film, and an RGB camera. (c) An example projector pattern and its corresponding captured image, exhibiting clear first-order diffraction.
• Figure 3: Image formation. (a) Camera response function and projector emission function. (b) Schematic diagram of image formation. (c) Depth dependency of the correspondence function $\psi$ for $m=1$ and a camera pixel $p$. (d) Spatially-varying correspondence map for depth 700 mm and wavelength 430 nm for the first-order diffractions $m=-1,1$.
• Figure 4: Binary decoding under dispersion. Simulated images under the binary-code patterns for (a) DSL and (b) conventional SL. (c) Intensity of a camera pixel with DSL and conventional SL. (d) Depth error of binary-code decoding for DSL and conventional SL with varying Gaussian noise.
• Figure 5: Hyperspectral imaging with scanline illuminations. Captured images under (a) white pattern and (b)&(c) scanline patterns. Narrow-band illumination over multiple columns is shown in (b), which originates from the first-order diffraction. (d) Pixel intensity with respect to varying scanline pattern index.