Neural Implicit Morphing of Face Images

TL;DR

The results of the experiments indicate that the proposed neural warping method is competitive with both classical and generative models under the lens of image quality and face-morphing detectors and presents a seamless blending of diverse faces not yet usual in the literature.

Abstract

Face morphing is a problem in computer graphics with numerous artistic and forensic applications. It is challenging due to variations in pose, lighting, gender, and ethnicity. This task consists of a warping for feature alignment and a blending for a seamless transition between the warped images. We propose to leverage coord-based neural networks to represent such warpings and blendings of face images. During training, we exploit the smoothness and flexibility of such networks by combining energy functionals employed in classical approaches without discretizations. Additionally, our method is time-dependent, allowing a continuous warping/blending of the images. During morphing inference, we need both direct and inverse transformations of the time-dependent warping. The first (second) is responsible for warping the target (source) image into the source (target) image. Our neural warping stores those maps in a single network dismissing the need for inverting them. The results of our experiments indicate that our method is competitive with both classical and generative models under the lens of image quality and face-morphing detectors. Aesthetically, the resulting images present a seamless blending of diverse faces not yet usual in the literature.

Figures (12)

• Figure 1: Schematic illustration of the neural warping $\textbf{T}$ being used to aligning the initial images $\text{I}_i$
• Figure 2: A neural warping $\textbf{T}$ continuously aligning two face images along time. We use $\textbf{T}$ to create their aligned warpings $\mathscr{I}_i$. The morphing $(1-t)\mathscr{I}_0+t\mathscr{I}_1$ was sampled at $t=0,0.25, 0.5, 0.75, 1$.
• Figure 3: Loss term impact experiment. From the left: results without the inverse, identity, data, and thin-plate constraints.
• Figure 4: Comparing different neural blendings of two faces $\text{I}_i$. Line 1/2 shows examples of cloning the half-space region of $\text{I}_1$ into $\text{I}_0$. In Column 1 we do not align the image landmarks, the remaining columns use our neural warping for the alignment. Column 2 uses $U \!\!=\! \!\hbox{Jac}\!\left( \mathscr{I}_1\right)$ and $\mathscr{I}^*\!\!=\!\!\mathscr{I}_0$ in the neural blending. Columns 3 and 4 applies the mixed and normal seamless clone respectively.
• Figure 5: Generative morphing. Line 1 presents a morphing between two faces using the generative morphing (neural warping + diffAE). Line 2 shows the results of diffAE using no alignment.