AI-Powered Facial Mask Removal Is Not Suitable For Biometric Identification

Abstract

Recently, crowd-sourced online criminal investigations have used generative-AI to enhance low-quality visual evidence. In one high-profile case, social-media users circulated an "AI-unmasked" image of a federal agent involved in a fatal shooting, fueling a wide-spread misidentification. In response to this and similar incidents, we conducted a large-scale analysis evaluating the efficacy and risks of commercial AI-powered facial unmasking, specifically assessing whether the resulting faces can be reliably matched to true identities.

Figures (3)

• Figure 1: A representative set of original unmasked faces (top) and the result of unmasking using Gemini to fill in the removal of the lower-half of the face. The value above each face is the biometric similarity (in $[-1,1]$) between the original and unmasked face. Larger values correspond to a higher facial similarity.
• Figure 2: Senator Elizabeth Warren's mask (left) is removed (middle) using ChatGPT (photo sources: commons.wikimedia.org).
• Figure 3: Distributions of biometric facial similarity scores: (A) pairs of images of the same identity (green), images of different identities from the same racial/gender group (red), and images of Doppelgangers (yellow); (B)-(D) pairs of a reference and unmasked image in which the lower-half was removed; and (E) pairs of a reference image and an image in which COVID masks were removed.