An Adaptive Screen-Space Meshing Approach for Normal Integration

TL;DR

This work introduces an adaptive surface triangulation in the image domain and afterwards performs the normal integration on a triangle mesh based on the curvature, finding that surface curvature can be computed from normals.

Abstract

Reconstructing surfaces from normals is a key component of photometric stereo. This work introduces an adaptive surface triangulation in the image domain and afterwards performs the normal integration on a triangle mesh. Our key insight is that surface curvature can be computed from normals. Based on the curvature, we identify flat areas and aggregate pixels into triangles. The approximation quality is controlled by a single user parameter facilitating a seamless generation of low- to high-resolution meshes. Compared to pixel grids, our triangle meshes adapt locally to surface details and allow for a sparser representation. Our new mesh-based formulation of the normal integration problem is strictly derived from discrete differential geometry and leads to well-conditioned linear systems. Results on real and synthetic data show that 10 to 100 times less vertices are required than pixels. Experiments suggest that this sparsity translates into a sublinear runtime in the number of pixels. For 64 MP normal maps, our meshing-first approach generates and integrates meshes in minutes while pixel-based approaches require hours just for the integration.

Figures (8)

• Figure 1: Our screen-space remeshing pipeline decimates smooth, featureless areas efficiently before the normal integration while preserving high-frequency details. Depicted results illustrate high, mid and low-resolution triangulations. Wireframes are rendered as vector graphics for closer examination.
• Figure 2: Left: Using the same number of variables, our meshing-first method achieves higher geometric fidelity. Right: Integration of an $8192^2$ normal map. The pixel-based approach requires around 3 hours while our meshing-first method generates comparable results in only 5 minutes.
• Figure 3: Normal integration results with (left) and without (right) the Gibbs phenomenon.
• Figure 4: Left: The projection $\phi$ maps points on the screen into three-dimensional space. Distances on the screen appear shorter than their three-dimensional counterparts (foreshortening). This is captured in the Jacobian $J$ of $\phi$. Right: Illustration on the relationship between the edge lengths and the user-defined approximation error $\epsilon$ as derived in Dunyach:2013.
• Figure 5: The meshing pipeline: We start by calculating curvature from the normal maps. This curvature is used to derive an optimal edge length. Then, an initial mesh is iteratively refined to move its edge lengths closer to the calculated optimum. The result is a sparse, isotropic mesh.