Partition-based Nonrigid Registration for 3D Face Model

TL;DR

The paper tackles nonrigid registration for 3D Morphable Models when there is a large disparity between a handcrafted template and captured faces. It introduces a partition-based approach that uses landmarks to divide the template, performs part-wise affine scaling via $A_i^T X_i=B_i^T$ with $X_i \in \mathbb{R}^{4\times3}$, and smooths boundaries to maintain mesh coherence; a normal-regulation term $E_n(T)$ is added to the energy and the optimization follows a nonrigid ICP framework. The extended energy becomes $E(T)=\alpha E_d(T)+\beta E_s(T)+\gamma E_l(T)+\eta E_n(T)$, enabling more reliable convergence. Experimental results show improved robustness to local minima and better warp quality compared to NICP, using partition schemes based on 11 landmarks or 68 Dlib landmarks, which enhances canonical 3DMM reconstruction across varied individuals and expressions.

Abstract

This paper presents a partition-based surface registration for 3D morphable model(3DMM). In the 3DMM, it often requires to warp a handcrafted template model into different captured models. The proposed method first utilizes the landmarks to partition the template model then scale each part and finally smooth the boundaries. This method is especially effective when the disparity between the template model and the target model is huge. The experiment result shows the method perform well than the traditional warp method and robust to the local minima.

Figures (5)

• Figure 1: Example of failure of the traditional registration-based method.
• Figure 2: a). The template model modified from paysan20093d. b). One target model captured by song2013accurate. c). Template partition by the handpicked points. d). Template partition by the Dlib dlib09 points.
• Figure 3: Steps of the partition-based algorithm
• Figure 4: Comparision of registration quality between NICP and ours.
• Figure 5: Registered 3D models of different facial expression by the 68 Dlib Partition strategy.