Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Noise-adapted Neural Operator for Robust Non-Line-of-Sight Imaging

Lianfang Wang, Kuilin Qin, Xueying Liu, Huibin Chang, Yong Wang, Yuping Duan

Abstract

This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Computational imaging, especially non-line-of-sight (NLOS) imaging, the extraction of information from obscured or hidden scenes is achieved through the utilization of indirect light signals resulting from multiple reflections or scattering. The inherently weak nature of these signals, coupled with their susceptibility to noise, necessitates the integration of physical processes to ensure accurate reconstruction. This paper presents a parameterized inverse problem framework tailored for large-scale linear problems in 3D imaging reconstruction. Initially, a noise estimation module is employed to adaptively assess the noise levels present in transient data. Subsequently, a parameterized neural operator is developed to approximate the inverse mapping, facilitating end-to-end rapid image reconstruction. Our 3D image reconstruction framework, grounded in operator learning, is constructed through deep algorithm unfolding, which not only provides commendable model interpretability but also enables dynamic adaptation to varying noise levels in the acquired data, thereby ensuring consistently robust and accurate reconstruction outcomes. Furthermore, we introduce a novel method for the fusion of global and local spatiotemporal data features. By integrating structural and detailed information, this method significantly enhances both accuracy and robustness. Comprehensive numerical experiments conducted on both simulated and real datasets substantiate the efficacy of the proposed method. It demonstrates remarkable performance with fast scanning data and sparse illumination point data, offering a viable solution for NLOS imaging in complex scenarios.

Noise-adapted Neural Operator for Robust Non-Line-of-Sight Imaging

Abstract

This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Computational imaging, especially non-line-of-sight (NLOS) imaging, the extraction of information from obscured or hidden scenes is achieved through the utilization of indirect light signals resulting from multiple reflections or scattering. The inherently weak nature of these signals, coupled with their susceptibility to noise, necessitates the integration of physical processes to ensure accurate reconstruction. This paper presents a parameterized inverse problem framework tailored for large-scale linear problems in 3D imaging reconstruction. Initially, a noise estimation module is employed to adaptively assess the noise levels present in transient data. Subsequently, a parameterized neural operator is developed to approximate the inverse mapping, facilitating end-to-end rapid image reconstruction. Our 3D image reconstruction framework, grounded in operator learning, is constructed through deep algorithm unfolding, which not only provides commendable model interpretability but also enables dynamic adaptation to varying noise levels in the acquired data, thereby ensuring consistently robust and accurate reconstruction outcomes. Furthermore, we introduce a novel method for the fusion of global and local spatiotemporal data features. By integrating structural and detailed information, this method significantly enhances both accuracy and robustness. Comprehensive numerical experiments conducted on both simulated and real datasets substantiate the efficacy of the proposed method. It demonstrates remarkable performance with fast scanning data and sparse illumination point data, offering a viable solution for NLOS imaging in complex scenarios.

Paper Structure

This paper contains 29 sections, 2 theorems, 35 equations, 10 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Forward model of NLOS
  4. Neural operators
  5. Neural-operator-based Reconstruction
  6. Mixed Noise Modeling
  7. Tikhonov Regularization Model
  8. Noise-adapted neural operator
  9. Network architecture
  10. Noise level estimation
  11. Noise-Adapted neural operator
  12. Global–Local Transient Lifting
  13. Complex Neural Operator Module
  14. Volumetric albedo projection
  15. Loss Function
  16. ...and 14 more sections

Key Result

Theorem 1

(Theorem 3.1 in he2023mgno) Let $\mathcal{X} \subset H^1(\Gamma)$ and $\mathcal{U} \subset H^{1}(\Omega)$ be Hilbert spaces. For any continuous operator $\mathcal{G}^*: \mathcal{X} \to \mathcal{U}$, any compact set $\mathcal{C} \subset \mathcal{X}$, and any $\varepsilon > 0$, there exists a neural o Specifically, the operator $\mathcal{G}$ can be realized as a neural network with $N$ neurons: whe

Figures (10)

  • Figure 1: Overview of confocal imaging and measurements. a. Histogram measured at the scanning points on the relay wall. b. Confocal NLOS system.
  • Figure 2: Flowchart of the proposed Noise-Adapted Neural Operator.
  • Figure 3: Evaluation of the noise level estimation module on simulated data. The left shows the noise range consistent with the training dataset, while the right illustrates a larger noise.
  • Figure 4: Reconstruction results from the test set of the simulated dataset for different comparison methods: the first and the third rows display the original simulated data, while the second and the fourth rows show the test results after noise is added to the dataset.
  • Figure 5: Visualization comparisons on hidden scenes from the public real-world data li2023nlost, where both maximum projection and 3D reconstruction are provided for different methods.
  • ...and 5 more figures

Theorems & Definitions (3)

  • Theorem 1
  • Theorem 2
  • proof