Depth Reconstruction with Neural Signed Distance Fields in Structured Light Systems

TL;DR

This work tackles depth reconstruction in monocular structured-light setups by employing a neural signed distance field (SDF) trained with differentiable rendering, leveraging the known radiance field from projected patterns to focus solely on geometry. The method introduces a volumetric rendering pipeline and a combination of losses, including a surface-color constraint and an Eikonal regularizer, enabling robust geometry with few projected patterns and allowing incremental optimization as new patterns are added. Key contributions include adapting a NeuS-style SDF to structured light, exploiting projected-pattern constraints to guide geometry, and demonstrating strong few-shot geometric performance with the potential for online pattern integration. The approach offers practical benefits for structured-light depth sensing, though it requires substantial compute time and careful handling of boundary artifacts; it paves the way for pattern-design strategies tailored to neural implicit representations.

Abstract

We introduce a novel depth estimation technique for multi-frame structured light setups using neural implicit representations of 3D space. Our approach employs a neural signed distance field (SDF), trained through self-supervised differentiable rendering. Unlike passive vision, where joint estimation of radiance and geometry fields is necessary, we capitalize on known radiance fields from projected patterns in structured light systems. This enables isolated optimization of the geometry field, ensuring convergence and network efficacy with fixed device positioning. To enhance geometric fidelity, we incorporate an additional color loss based on object surfaces during training. Real-world experiments demonstrate our method's superiority in geometric performance for few-shot scenarios, while achieving comparable results with increased pattern availability.

Figures (8)

• Figure 1: A schematic diagram of a monocular structured light system. Once a correspondence between the camera and the projector is established, the depth information can be computed.
• Figure 2: The general framework of our method for depth reconstruction using neural signed distance fields. Given a pixel from the camera space, the color/intensity is rendered using a combination of the neural signed distance field and the volumetric rendering process. The rendered color/intensity is then compared with the captured image to train the neural network.
• Figure 3: Distinguishing between Rendered Color Loss $\mathcal{L}_{rc}$ and Surface Color Loss $\mathcal{L}_{sc}$. $\mathcal{L}_{sc}$ enforces a singular peak along the projection ray for geometry consistency, while $\mathcal{L}_{rc}$ emphasizes accurate color in rendered images. In an example, correct rendered color (upper) contrasts with flawed surface color (lower), highlighting geometry inaccuracies in wave-like regions.
• Figure 4: The experiments setting for our data collection. Six scenarios with different objects are used for our data collection.
• Figure 5: Visualization of error maps and point clouds in two scenes. We also append the patterns used by each method at the bottom of each method.