Deep Learning-Based Quasi-Conformal Surface Registration for Partial 3D Faces Applied to Facial Recognition

TL;DR

The paper tackles the challenge of 3D facial recognition from partial data by integrating quasi-conformal geometry with deep learning. It introduces a curvature-informed Landmark Detection Network (LD-Net) to locate facial features on partial surfaces, and a Coefficients Prediction Network (CP-Net) that estimates Beltrami coefficients $\mu$ to drive a Beltrami Solver Network (BSNet) that reconstructs a bijective quasi-conformal mapping to a template face. The method leverages conformal parameterization via DNCP and LSQC to obtain low-distortion 2D representations, then jointly optimizes landmark accuracy, distortion, and curvature-based overlap in a unifying loss $\mathcal{L}_{Reg}$. Experiments on synthetic partial faces and the LFW dataset demonstrate robust landmark detection, accurate partial-to-template registration with invertible mappings, and competitive recognition performance (e.g., ~$91.7\%$ accuracy) with strong statistical separation, highlighting potential for practical biometric and HCI applications. The approach provides dense point-wise correspondences between partial faces, enabling precise geometric comparison and reliable recognition under occlusion and partial visibility.

Abstract

3D face registration is an important process in which a 3D face model is aligned and mapped to a template face. However, the task of 3D face registration becomes particularly challenging when dealing with partial face data, where only limited facial information is available. To address this challenge, this paper presents a novel deep learning-based approach that combines quasi-conformal geometry with deep neural networks for partial face registration. The proposed framework begins with a Landmark Detection Network that utilizes curvature information to detect the presence of facial features and estimate their corresponding coordinates. These facial landmark features serve as essential guidance for the registration process. To establish a dense correspondence between the partial face and the template surface, a registration network based on quasiconformal theories is employed. The registration network establishes a bijective quasiconformal surface mapping aligning corresponding partial faces based on detected landmarks and curvature values. It consists of the Coefficients Prediction Network, which outputs the optimal Beltrami coefficient representing the surface mapping. The Beltrami coefficient quantifies the local geometric distortion of the mapping. By controlling the magnitude of the Beltrami coefficient through a suitable activation function, the bijectivity and geometric distortion of the mapping can be controlled. The Beltrami coefficient is then fed into the Beltrami solver network to reconstruct the corresponding mapping. The surface registration enables the acquisition of corresponding regions and the establishment of point-wise correspondence between different partial faces, facilitating precise shape comparison through the evaluation of point-wise geometric differences at these corresponding regions. Experimental results demonstrate the effectiveness of the proposed method.

Figures (15)

• Figure 1: The illustration of how Beltrami coefficient $\mu$ determine the conformal distortion.
• Figure 2: Demonstration of the surface matching problem. $S_1$ and $S_2$ are two partial faces with similar parts. $\Omega_1$ and $\Omega_2$ are the intercepted parts and labeled in magenta. The goal is to find a bijective mapping $f$ such that the regional correspondence can be constructed for $\Omega_1$ and $\Omega_2$.
• Figure 3: Demonstration of the strategy for solving partial surface correspondence problem. $S_1$ and $S_2$ are two partial faces in cyan and magenta, respectively. $T$ is the template face in gray. $f_1$ and $f_2$ are bijective mappings.
• Figure 4: Demonstration of the framework for registration. The first row demonstrates how we generate a quasi-conformal mapping based on the input face mesh. Then, we demonstrate how the mapping registers an input face mesh to the template face.
• Figure 5: The flow on the left demonstrates the parameterization by discrete natural conformal parameterization(DNCP) and modified by least-squares quasi-conformal map (LSQC). The chart on the right shows the angle distortion which is small.

Theorems & Definitions (1)

• Theorem 1