Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalizable Non-Line-of-Sight Imaging with Learnable Physical Priors

Shida Sun, Yue Li, Yueyi Zhang, Zhiwei Xiong

TL;DR

Experimental validations demonstrate that the proposed approach, only trained on synthetic data, exhibits the capability to seamlessly generalize across various real-world datasets captured by different imaging systems and characterized by low SNRs.

Abstract

Non-line-of-sight (NLOS) imaging, recovering the hidden volume from indirect reflections, has attracted increasing attention due to its potential applications. Despite promising results, existing NLOS reconstruction approaches are constrained by the reliance on empirical physical priors, e.g., single fixed path compensation. Moreover, these approaches still possess limited generalization ability, particularly when dealing with scenes at a low signal-to-noise ratio (SNR). To overcome the above problems, we introduce a novel learning-based solution, comprising two key designs: Learnable Path Compensation (LPC) and Adaptive Phasor Field (APF). The LPC applies tailored path compensation coefficients to adapt to different objects in the scene, effectively reducing light wave attenuation, especially in distant regions. Meanwhile, the APF learns the precise Gaussian window of the illumination function for the phasor field, dynamically selecting the relevant spectrum band of the transient measurement. Experimental validations demonstrate that our proposed approach, only trained on synthetic data, exhibits the capability to seamlessly generalize across various real-world datasets captured by different imaging systems and characterized by low SNRs.

Generalizable Non-Line-of-Sight Imaging with Learnable Physical Priors

TL;DR

Experimental validations demonstrate that the proposed approach, only trained on synthetic data, exhibits the capability to seamlessly generalize across various real-world datasets captured by different imaging systems and characterized by low SNRs.

Abstract

Non-line-of-sight (NLOS) imaging, recovering the hidden volume from indirect reflections, has attracted increasing attention due to its potential applications. Despite promising results, existing NLOS reconstruction approaches are constrained by the reliance on empirical physical priors, e.g., single fixed path compensation. Moreover, these approaches still possess limited generalization ability, particularly when dealing with scenes at a low signal-to-noise ratio (SNR). To overcome the above problems, we introduce a novel learning-based solution, comprising two key designs: Learnable Path Compensation (LPC) and Adaptive Phasor Field (APF). The LPC applies tailored path compensation coefficients to adapt to different objects in the scene, effectively reducing light wave attenuation, especially in distant regions. Meanwhile, the APF learns the precise Gaussian window of the illumination function for the phasor field, dynamically selecting the relevant spectrum band of the transient measurement. Experimental validations demonstrate that our proposed approach, only trained on synthetic data, exhibits the capability to seamlessly generalize across various real-world datasets captured by different imaging systems and characterized by low SNRs.
Paper Structure (17 sections, 11 equations, 9 figures, 2 tables)

This paper contains 17 sections, 11 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional Approaches
  4. Learning-based Approaches
  5. Methodology
  6. Imaging Formulation
  7. Overview
  8. Learning to Compensate Radiometric Intensity Fall-off
  9. Denoising with Adaptive Phasor Field
  10. Loss Function
  11. Experimental Results
  12. Baselines and Datasets
  13. Implementation Details and Metrics
  14. Comparison on Synthetic Data
  15. Comparison on Real-world Data
  16. ...and 2 more sections

Figures (9)

  • Figure 1: (a) An overview of the NLOS imaging system, including objects with distinct surface materials. (b) Reconstructed images from our method and RSD liu2019non with different compensation coefficients. Near to Far: Dragon, Bookshelf, Statue.
  • Figure 2: An overview of our proposed approach. Given the transient measurements as input, the approach generates the albedo volume, intensity image, and depth map.
  • Figure 3: The pipeline of the LPC.
  • Figure 4: The pipeline of the APF. The module predicts the illumination function with an appropriate bandwidth to compensate for the noisy transient features, outputting clean, denoised features.
  • Figure 5: Intensity results recovered by different approaches on the Seen test set. GT means ground truth of the intensity images.
  • ...and 4 more figures