Dual-branch Graph Feature Learning for NLOS Imaging

TL;DR

Non-line-of-sight imaging faces high computational demands and a coupled albedo-depth reconstruction problem. This work introduces DG-NLOS, a dual-branch graph neural network that decouples albedo and depth reconstruction and converts dense 3D grid measurements into sparse graph features for efficient processing. It features a graph block, channel fusion module, and a decoupled two-stage optimization with a multi-scale loss, achieving state-of-the-art performance on both synthetic and real data with reduced memory usage. The approach enhances practical NLOS imaging by enabling high-quality texture and geometry recovery under challenging noise and real-world conditions.

Abstract

The domain of non-line-of-sight (NLOS) imaging is advancing rapidly, offering the capability to reveal occluded scenes that are not directly visible. However, contemporary NLOS systems face several significant challenges: (1) The computational and storage requirements are profound due to the inherent three-dimensional grid data structure, which restricts practical application. (2) The simultaneous reconstruction of albedo and depth information requires a delicate balance using hyperparameters in the loss function, rendering the concurrent reconstruction of texture and depth information difficult. This paper introduces the innovative methodology, \xnet, which integrates an albedo-focused reconstruction branch dedicated to albedo information recovery and a depth-focused reconstruction branch that extracts geometrical structure, to overcome these obstacles. The dual-branch framework segregates content delivery to the respective reconstructions, thereby enhancing the quality of the retrieved data. To our knowledge, we are the first to employ the GNN as a fundamental component to transform dense NLOS grid data into sparse structural features for efficient reconstruction. Comprehensive experiments demonstrate that our method attains the highest level of performance among existing methods across synthetic and real data. https://github.com/Nicholassu/DG-NLOS.

Figures (9)

• Figure 1: Various types of Deep NLOS reconstruction. Blue arrows denote the transform with physical models. The two real scene reconstructions are compared, where green rectangles highlight the superior performance of DG-NLOS.
• Figure 2: A schematic diagram of the C-NLOS system.
• Figure 3: (a) Structure of the two-stage learning pipeline: The loss functions including Albedo and Depth are calculated in triple scales, respectively. Finally, the output voxels of two branches are combined for the optimized reconstruction. (b) Structure of graph module.(c) Structure of channel fusion. Each module adopts the Resnet skip connection mechanism.
• Figure 4: Graph construction and selection. Cyan edges connect neighbor vertices, while red edges denote negative connections. From (a) to (b), original neighbor vertices $7$ and $5$ are removed due to closer distance with negative vertices.
• Figure 5: Illustration of joint optimization and decoupling optimization. $\boldsymbol{\theta}, {\boldsymbol{\eta}}$ and $\boldsymbol{\mu}$ are the optimized parameters for joint optimization, albedo, and depth branches, respectively.