Domain Reduction Strategy for Non Line of Sight Imaging

TL;DR

This work tackles non-line-of-sight imaging under general relay-wall geometries and sparse sampling by introducing an optimization-based pipeline augmented with a domain reduction strategy. Transients are modeled as a linear superposition of point-wise propagations from continuously sampled hidden-space points, with per-voxel albedo $\rho$ and normal $\boldsymbol{n}$ optimized to match measurements, and a coarse-to-fine domain reduction that prunes empty regions to drastically reduce computations. The method demonstrates high-quality recovery of albedo and surface normals across confocal, non-confocal, and non-planar scenarios, achieving around a minute reconstruction time on a single consumer GPU even with $128 \times 128$ outputs and sparse $32 \times 32$ scans. Overall, the domain-reduction approach preserves the generality of optimization-based NLOS methods while delivering substantial efficiency gains, broadening practical applicability for real-world NLOS imaging tasks.

Abstract

This paper presents a novel optimization-based method for non-line-of-sight (NLOS) imaging that aims to reconstruct hidden scenes under general setups with significantly reduced reconstruction time. In NLOS imaging, the visible surfaces of the target objects are notably sparse. To mitigate unnecessary computations arising from empty regions, we design our method to render the transients through partial propagations from a continuously sampled set of points from the hidden space. Our method is capable of accurately and efficiently modeling the view-dependent reflectance using surface normals, which enables us to obtain surface geometry as well as albedo. In this pipeline, we propose a novel domain reduction strategy to eliminate superfluous computations in empty regions. During the optimization process, our domain reduction procedure periodically prunes the empty regions from our sampling domain in a coarse-to-fine manner, leading to substantial improvement in efficiency. We demonstrate the effectiveness of our method in various NLOS scenarios with sparse scanning patterns. Experiments conducted on both synthetic and real-world data support the efficacy in general NLOS scenarios, and the improved efficiency of our method compared to the previous optimization-based solutions. Our code is available at https://github.com/hyunbo9/domain-reduction-strategy.

Figures (15)

• Figure 1: (a) A common NLOS scanning system. A laser and a time-resolved sensor illuminate and scan the relay wall. (b) Our method reconstructs both albedo and surface normal of the hidden objects in general scenarios, including non-confocal, non-planar relay walls and sparse sampling. (c) Our domain reduction gradually prunes empty regions in a coarse-to-fine manner, achieving significant efficiency improvement.
• Figure 2: (a) The overview of our reconstruction scheme. We divide hidden space to a grid shape and assign albedo $\rho$ and surface normal $\hbox{\boldmath$n$}$ for each vertex. Input points are randomly sampled from the hidden space. Then the light propagation from each point $G_\mathbf{p}$ is computed, of which superposition is used as predicted transients. The variables are optimized by minimizing the L2 distance. Our domain is gradually reduced by pruning the empty regions during the optimization. (b) The computation of light propagation function from each point $\mathbf{p}$. The arrival time at scan point $\mathbf{s}$ is first identified, and then the fall-off terms $\Phi_\mathbf{p}$ and $\Upsilon_\mathbf{p}$ are computed using $\mathbf{p}$, $\mathbf{l}$, $\mathbf{\mathbf{s}}$ and $\hbox{\boldmath$n$}(\mathbf{p})$.
• Figure 3: Reconstructed albedo on the transients with a confocal $32 \times 32$ scan system and a planar relay wall. Our method achieves the highest reconstruction quality.
• Figure 4: Comparison of the reconstructed normal map with DLCT young2020dlct on the transients with a confocal $32 \times 32$ scan system and a planar relay wall. Our method accurately reconstructs surface geometry, whereas DLCT only produces coarse structures.
• Figure 5: Results on various setting. (a) Results on the real-world $32 \times 32$ transients of the NT instance lindell2019fk, measured with a non-planar relay wall. (b) Results on non-confocal ZNLOS Bunny. (c) Results on Bunny with various sampling resolutions. Our method consistently delivers high-quality results in various scanning scenarios.