Optimized Sampling for Non-Line-of-Sight Imaging Using Modified Fast Fourier Transforms

TL;DR

This work tackles the rigidity of FFT-based NLOS imaging pipelines that assume uniform sampling on both the relay surface and reconstruction grid. By introducing Scaled RSD (SRSD) and Non-Uniform RSD (NURSD), it leverages Scaling Factors and Non-Uniform Fast Fourier Transforms to expand reconstruction volume and flexibility without increasing computational complexity. The phasor-field framework underpins the methods, enabling memory-efficient, perspective-aware 3D reconstructions from irregular relay surfaces and SPAD arrays. The results demonstrate comparable reconstruction quality to traditional FFT-based methods while enabling larger volumes, nonuniform acquisitions, and data compression, with practical implications for real-world NLOS imaging deployments. The work lays a foundation for adaptive sampling and noise-aware processing to further improve robustness and efficiency.

Abstract

Non-line-of-Sight (NLOS) imaging systems collect light at a diffuse relay surface and input this measurement into computational algorithms that output a 3D volumetric reconstruction. These algorithms utilize the Fast Fourier Transform (FFT) to accelerate the reconstruction process but require both input and output to be sampled spatially with uniform grids. However, the geometry of NLOS imaging inherently results in non-uniform sampling on the relay surface when using multi-pixel detector arrays, even though such arrays significantly reduce acquisition times. Furthermore, using these arrays increases the data rate required for sensor readout, posing challenges for real-world deployment. In this work, we utilize the phasor field framework to demonstrate that existing NLOS imaging setups typically oversample the relay surface spatially, explaining why the measurement can be compressed without significantly sacrificing reconstruction quality. This enables us to utilize the Non-Uniform Fast Fourier Transform (NUFFT) to reconstruct from sparse measurements acquired from irregularly sampled relay surfaces of arbitrary shapes. Furthermore, we utilize the NUFFT to reconstruct at arbitrary locations in the hidden volume, ensuring flexible sampling schemes for both the input and output. Finally, we utilize the Scaled Fast Fourier Transform (SFFT) to reconstruct larger volumes without increasing the number of samples stored in memory. All algorithms introduced in this paper preserve the computational complexity of FFT-based methods, ensuring scalability for practical NLOS imaging applications.

Figures (21)

• Figure 1: These Modified Fast Fourier Transforms can convert between time $t$ and frequency $f$ with flexible sampling schemes without increasing computational complexity. $\Delta$ is used for uniform spacing, while non-uniform samples are indexed by subscript $\ell$.
• Figure 2: Showing the top view of the reconstruction volume when reconstructing with the RSD algorithm (left) and the SRSD algorithm (right). The dotted green lines demonstrate the increase in volume as we increase $z$, when the lateral field of view (FOV) is fixed. The RSD algorithm requires zero padding and increasing the side length, while the SRSD increases the lateral FOV without increasing the number of pixels.
• Figure 3: (A) The relay surface is sparsely sampled at the green locations and interpolated to a dense grid denoted by the red locations. Reconstructing the hidden scene using the Standard RSD algorithm on the interpolated grid generates an output with minimal loss in reconstruction quality. (B) The NURSD-2 algorithm can generate reconstructions where the output i.e. the voxel grid is processed with flexible sampling schemes. (C) The NURSD-2 algorithm can generate reconstructions where both the input and output are discretized with flexible sampling schemes.
• Figure 4: Column 1 presents the hidden scene as viewed from the relay surface. Columns 2, 3, and 4 display reconstructions using the standard RSD and the scaled RSD, where a max filter is applied along the depth to generate 2D images.The average reconstruction time in seconds is shown in white text at the bottom center of each image, while the black text below it indicates the dimension-wise number of voxels (X x Y x Z) used for the reconstruction volume. Column 5 provides a top view of the reconstructed volumes for each algorithm, illustrating that both the scaled RSD and RSD-2 reconstruct larger lateral areas. Moreover, the scaled RSD captures perspective effects that are absent in the standard RSD.
• Figure 5: Column 1 shows the image of the hidden scene. Columns 2 and 3 display the standard RSD reconstruction and the scaled RSD reconstruction, respectively, after applying a max filter operation. Row 1: Two letter T's of the same size are shown, with the T in the middle appearing smaller in the SRSD reconstruction due to its greater depth. Row 2: Letters T, W, and U of different sizes are arranged at varying depths so that they appear the same size when viewed from the relay wall. The numbers at the bottom center indicate the time in seconds needed to reconstruct each image.