Discontinuity-preserving Normal Integration with Auxiliary Edges

TL;DR

This work addresses depth reconstruction from normal maps in the presence of surface discontinuities caused by occlusions. It introduces a discrete graph with auxiliary edges that explicitly represent jumps, and an iterative, sparsity-regularized optimization that combines IRLS with gradient filtering to recover both the depth and discontinuities. The method solves a weighted least squares depth problem while progressively enforcing sparse, meaningful gradient edits on auxiliary edges, yielding accurate small and large discontinuities that prior methods struggle to preserve. The approach enhances 3D reconstruction from normals, enabling more faithful surface details in applications such as photometric stereo and single-image 3D modeling.

Abstract

Many surface reconstruction methods incorporate normal integration, which is a process to obtain a depth map from surface gradients. In this process, the input may represent a surface with discontinuities, e.g., due to self-occlusion. To reconstruct an accurate depth map from the input normal map, hidden surface gradients occurring from the jumps must be handled. To model these jumps correctly, we design a novel discretization scheme for the domain of normal integration. Our key idea is to introduce auxiliary edges, which bridge between piecewise-smooth patches in the domain so that the magnitude of hidden jumps can be explicitly expressed. Using the auxiliary edges, we design a novel algorithm to optimize the discontinuity and the depth map from the input normal map. Our method optimizes discontinuities by using a combination of iterative re-weighted least squares and iterative filtering of the jump magnitudes on auxiliary edges to provide strong sparsity regularization. Compared to previous discontinuity-preserving normal integration methods, which model the magnitudes of jumps only implicitly, our method reconstructs subtle discontinuities accurately thanks to our explicit representation of jumps allowing for strong sparsity regularization.

Figures (10)

• Figure 1: Our normal integration recovers a depth map from an input normal map while preserving surface discontinuities. Compared to the state-of-the-art discontinuity-preserving normal integration method, BiNI cao2022bilateral, our method recovers small discontinuities accurately.
• Figure 2: Construction of our graph for normal integration. For each pixel in the input gradient map, we construct a quadrilateral with four vertices for depth and their edges $\mathcal{E}_v$ for directional derivatives. The quadrilaterals are bridged with auxiliary edges $\mathcal{E}_a$ that model gradients across the discontinuity.
• Figure 3: Construction of depth map from the normal integration results on the graph. The depth values on the four vertices of each quadrilateral are averaged and assigned to a depth pixel.
• Figure 4: Visualization of $\mathbf{g}'$ and $\mathbf{d}$ across iterative optimization. The gradients $\mathbf{g}'$ on auxiliary edges are visualized as color; bright color means higher magnitude. The depth $\mathbf{d}$ is shown as the height on the surface. The numbers at the bottom show the number of iterations $n$.
• Figure 5: Qualitative comparison with previous normal integration methods using the DiLiGeNT dataset. Our method can reconstruct challenging and small discontinuities as well as large discontinuities that are handled by BiNI cao2022bilateral.