Reconstructing Topology-Consistent Face Mesh by Volume Rendering from Multi-View Images

TL;DR

The paper tackles reconstructing topology-consistent face meshes from multi-view images by marrying an explicit artist-designed mesh template with neural volume rendering. It derives a differentiable density field from the mesh via distance-to-surface mapping and encodes appearance with tri-planes, enabling end-to-end optimization of geometry and texture while preserving topology. A five-term loss and a progressive training scheme guide geometry initialization, tri-plane refinement, and joint optimization, achieving higher rendering fidelity and robust topology preservation without pretraining or face priors. Experiments on a multi-view face dataset demonstrate improved reconstruction accuracy and rendering quality, including robustness under sparse views, highlighting the method’s potential for production pipelines.

Abstract

Industrial 3D face assets creation typically reconstructs topology-consistent face meshes from multi-view images for downstream production. However, high-quality reconstruction usually requires manual processing or specific capture settings. Recently NeRF has shown great advantages in 3D reconstruction, by representing scenes as density and radiance fields and utilizing neural volume rendering for novel view synthesis. Inspired by this, we introduce a novel method which combines explicit mesh with neural volume rendering to optimize geometry of an artist-made template face mesh from multi-view images while keeping the topology unchanged. Our method derives density fields from meshes using distance fields as an intermediary and encodes radiance field in compact tri-planes. To improve convergence, several adaptions tailored for meshes are introduced to the volume rendering. Experiments demonstrate that our method achieves superior reconstruction quality compared to previous approaches, validating the feasibility of integrating mesh and neural volume rendering.

Figures (7)

• Figure 1: Our method takes multi-view images (a) as input and reconstructs face mesh (d) accurately while preserving the topology of the artist-designed template (c). We also show a raw face scan with arbitrary topology(b).
• Figure 2: The pipeline of our method. We introduce mesh volume rendering which transforms point-to-mesh distance into density, utilize tri-planes to store view-dependent face appearances, and generate high-quality images via volume rendering, thus enabling image supervision on the mesh geometry.
• Figure 3: Qualitative comparison on reconstruction accuracy.
• Figure 4: Qualitative analysis on topology consistency of the reconstructed meshes.
• Figure 5: Qualitative comparison with NeuS on rendering quality.