Facial Demorphing via Identity Preserving Image Decomposition

TL;DR

The paper tackles morph attacks in face recognition by proposing a reference-free demorphing framework built on identity-preserving image decomposition. It introduces a two-stage architecture with a decomposer and a merger that split morphed inputs into multiple unintelligible components and then reconstruct bonafides through learned weighting, extending this to demorphing with a cross-road loss to recover underlying identities. Across CASIA-WebFace, SMDD, and AMSL datasets, the method achieves high reconstruction fidelity (FID ≈ 0.18–0.22, SSIM ≈ 0.99, PSNR ≈ 39–44) and strong restoration accuracy for bonafides (often >97%), while revealing limited identity leakage in individual components. The approach demonstrates both robust bonafide recovery and potential applicability as a morph-audit or MAD tool, while primarily addressing the relatively constrained scenario-1 setting and leaving expansions to more challenging scenarios for future work.

Abstract

A face morph is created by combining the face images usually pertaining to two distinct identities. The goal is to generate an image that can be matched with two identities thereby undermining the security of a face recognition system. To deal with this problem, several morph attack detection techniques have been developed. But these methods do not extract any information about the underlying bonafides used to create them. Demorphing addresses this limitation. However, current demorphing techniques are mostly reference-based, i.e, they need an image of one of the identities to recover the other. In this work, we treat demorphing as an ill-posed decomposition problem. We propose a novel method that is reference-free and recovers the bonafides with high accuracy. Our method decomposes the morph into several identity-preserving feature components. A merger network then weighs and combines these components to recover the bonafides. Our method is observed to reconstruct high-quality bonafides in terms of definition and fidelity. Experiments on the CASIA-WebFace, SMDD and AMSL datasets demonstrate the effectiveness of our method.

Figures (5)

• Figure 1: Image Decomposition for Demorphing. A decomposer network decomposes the morphed image into its components. The components are then individually picked and weighed by the merger to reconstruct the bonafides.
• Figure 2: Architecture of the decomposition pipeline. A multi-decoder UNet, $\mathcal{D}$, consisting of an encoder, $\mathcal{D}_{enc}$, and $k$ decoders, $\mathcal{D}_{dec}$, decomposes the input, $\mathcal{I}$, into its components. These components are weighed according to $w$ and a multi-encoder UNet, $\mathcal{M}$, consisting of $k$ encoders, $\mathcal{M}_{enc}$, and a decoder, $\mathcal{M}_{dec}$, reconstructs the input using the components.
• Figure 3: Identity decomposition on sample images from the SMDD bonafide and CASIA-WebFace datasets. (a,c) The first column has the real faces from the datasets, the next three columns are the components extracted by $\mathcal{D}$ followed by the image reconstructed by $\mathcal{M}$. The values under the image denote the similarity score to the original image where 'N/A' indicates that a face is not found. (b,d) Attempting to recover identity using only one component. $\mathcal{M}$ only recovers the face iff all the components are present making the individual components obsolete without others.
• Figure 4: Reference Free Demorphing on SMDD and AMSL datasets. $\mathcal{I},\mathcal{B}_1,\mathcal{B}_2$ are the morph and bonafides, respectively. $\mathcal{I}_1,\mathcal{I}_2,\mathcal{I}_3$ are the decomposed components corresponding to $\mathcal{I}$, and $\mathcal{O}_1,\mathcal{O}_2$ denote the outputs produced by the model. We list the AdaFace similarity score ref22 between the morph and bonafide below $\mathcal{B}_1$ and $\mathcal{B}_2$. 'N/A' represents face-not-found and the scores below $\mathcal{O}_1$ and $\mathcal{O}_2$ are AdaFace similarity scores between the outputs and their corresponding bonafides.
• Figure 5: Our method produces near duplicates of the input when presented with a non-morph. This observation can be used to detect morphs.