Single-pixel 3D imaging based on fusion temporal data of single photon detector and millimeter-wave radar

TL;DR

This work tackles symmetry blur in single-pixel 3D imaging by fusing temporal data from a single-pixel SPAD with a millimeter-wave radar to form a combined 1D histogram $H$. An artificial neural network learns the inverse mapping $F^{-1}$ from the fused histogram to the 3D scene $I$, using training pairs of fused histograms and ground-truth depth maps. Both simulations and experiments show that fusion data significantly improves reconstruction quality, outperforming SPD-only and radar-only approaches, and effectively resolves symmetry ambiguities without requiring a background. The approach offers a low-cost, all-weather pathway for robust 3D imaging with potential applications in autonomous navigation and security.

Abstract

Recently, there has been increased attention towards 3D imaging using single-pixel single-photon detection (also known as temporal data) due to its potential advantages in terms of cost and power efficiency. However, to eliminate the symmetry blur in the reconstructed images, a fixed background is required. This paper proposes a fusion-data-based 3D imaging method that utilizes a single-pixel single-photon detector and a millimeter-wave radar to capture temporal histograms of a scene from multiple perspectives. Subsequently, the 3D information can be reconstructed from the one-dimensional fusion temporal data by using Artificial Neural Network (ANN). Both the simulation and experimental results demonstrate that our fusion method effectively eliminates symmetry blur and improves the quality of the reconstructed images.

Figures (5)

• Figure 1: 3D imaging with fusion data. (a) Data acquisition process. (b) 3D images recovery process. In 3D images recovery process, the ANN training is performed only once, then the MLP algorithm can directly reconstructs the 3D image from the temporal histogram. The Radar represents the millimeter-wave radar. The human moves in an empty room
• Figure 2: Single-pixel single-photon detector imaging simulation result. (a) Single-pixel 3D imaging symmetry blur result due to lack of background information, (b) 3D imaging without symmetry blur in the background
• Figure 3: Simulation and reconstruction results of the fusion method. (a)-(c) Images recovered using a fused data-based 3D image reconstruction algorithm. (d)-(f) Images recovered using only single photon data. The first column shows fused temporal histograms generated by simulation [rows (a)–(c)] or histograms with only single-photon data [rows (d)–(f)], The second column shows the ground truth depth maps generated by simulation, and the third column shows the reconstructed images of the time histogram recovered by the MLP algorithm
• Figure 4: Schematic diagram of the experimental system and experimental scene. (a) Schematic of the layout of the 1550nm fusion-data-based single-photon single-pixel 3D imaging system which comprises a supercontinuum laser source, an SNSNP, a TDC module, and Optical lens system, a depth camera, a millimeter-wave radar, and a laptop. (b) experimental scene with a person. (c) Schematic diagram of an optical lens system
• Figure 5: Experiment results. (a)-(d) Imaging result of different people and objects. In each sub-figure, the Histogram, the ground-truth depth maps and the retrieved images are shown from left to right. Moreover, the reconstructed images with fusion data, only single photon data, only millimeter-wave data are shown from top to bottom