GLVD: Guided Learned Vertex Descent

TL;DR

GLVD tackles high-fidelity 3D face reconstruction from few-shot images by fusing per-vertex neural field optimization with global structural guidance from dynamically predicted 3D keypoints. The method encodes vertex updates relative to a sparse set of keypoints, enabling iterative refinement without dense 3D supervision and without a fixed morphable-model prior. Evaluations on single-view benchmarks show state-of-the-art performance, while multi-view results remain competitive, all with significantly faster inference than traditional optimization approaches. By pretraining on SDF tasks and leveraging a canonical space, GLVD achieves robust convergence and accurate geometry with relatively low computational cost, making it practical for real-time or near-real-time applications in AR/VR and graphics.

Abstract

Existing 3D face modeling methods usually depend on 3D Morphable Models, which inherently constrain the representation capacity to fixed shape priors. Optimization-based approaches offer high-quality reconstructions but tend to be computationally expensive. In this work, we introduce GLVD, a hybrid method for 3D face reconstruction from few-shot images that extends Learned Vertex Descent (LVD) by integrating per-vertex neural field optimization with global structural guidance from dynamically predicted 3D keypoints. By incorporating relative spatial encoding, GLVD iteratively refines mesh vertices without requiring dense 3D supervision. This enables expressive and adaptable geometry reconstruction while maintaining computational efficiency. GLVD achieves state-of-the-art performance in single-view settings and remains highly competitive in multi-view scenarios, all while substantially reducing inference time.

Figures (13)

• Figure 1: Qualitative results for two in-the-wild subjects reconstructed using GLVD .
• Figure 2: Overview of GLVD . Given one or more input images, each paired with a head mask and calibrated camera parameters, the method reconstructs a 3D face mesh through two branches. (1) The 3D Keypoint Branch predicts a set of facial keypoints by extracting localized image features and estimating their 3D displacements iteratively. (2) The 3D Vertex Branch refines the full-face geometry by leveraging these keypoints to encode relative spatial information for each surface vertex. This branch extracts pixel-aligned features and predicts vertex-wise displacements in an iterative optimization process.
• Figure 3: Qualitative results on two subjects of the H3DS dataset, for LVD corona2022learned, SIRA++caselles2025implicit, JIFF cao2022jiff, and GLVD , with an increasing number of input views.
• Figure 4: Qualitative results on the 3DFAW dataset for a single input image. Each 3D reconstructed face is accompanied by a heatmap, where reddish areas indicate larger errors in mm.
• Figure 5: Qualitative results on the 3DFAW dataset for three input images. Each 3D reconstructed face is accompanied by a heatmap, where reddish areas indicate larger errors in mm.