Detecting Near-Duplicate Face Images

TL;DR

The paper tackles detecting near-duplicate face images by modeling their relationships with Image Phylogeny Trees (IPTs) and assembling these into Image Phylogeny Forests (IPFs). It combines a locally-scaled spectral clustering stage, which uses fused pixel, PRNU, and face-descriptor features to group near-duplicates, with a graph neural network-based node embedding and a PRNU-driven link-prediction stage to recover hierarchical, parent-child relationships. The approach is demonstrated to be robust across unseen transformations, modalities, configurations, and varying IPT sizes, achieving substantial improvements over baselines (up to approximately 42% in IPF reconstruction accuracy) and generalizing to biometric and natural-scene images. The work offers a domain-agnostic framework for provenance analysis in digital imagery, with potential implications for biometric security and copyright enforcement, and it reports strong performance across demographic groups and diverse datasets.

Abstract

Near-duplicate images are often generated when applying repeated photometric and geometric transformations that produce imperceptible variants of the original image. Consequently, a deluge of near-duplicates can be circulated online posing copyright infringement concerns. The concerns are more severe when biometric data is altered through such nuanced transformations. In this work, we address the challenge of near-duplicate detection in face images by, firstly, identifying the original image from a set of near-duplicates and, secondly, deducing the relationship between the original image and the near-duplicates. We construct a tree-like structure, called an Image Phylogeny Tree (IPT) using a graph-theoretic approach to estimate the relationship, i.e., determine the sequence in which they have been generated. We further extend our method to create an ensemble of IPTs known as Image Phylogeny Forests (IPFs). We rigorously evaluate our method to demonstrate robustness across other modalities, unseen transformations by latest generative models and IPT configurations, thereby significantly advancing the state-of-the-art performance by 42% on IPF reconstruction accuracy.

Figures (16)

• Figure 1: Examples of near-duplicates available online illustrating celebrity face images modified through (top row) Photoshop, (middle row) FaceApp and (bottom row) MakeApp filters. Image courtesy imagecourtesy.
• Figure 2: Two examples of near-duplicates generated using a sequence of photometric transformations — Brightness adjustment, Gamma transformation, Gaussian smoothing and Median filtering. In the top row, the original image is '2', and in the bottom row, the original image is '3'. Visual inspection cannot trivially determine the sequence in which the images were modified.
• Figure 3: Two examples of image phylogeny trees (IPTs) corresponding to the two sets of near-duplicate images presented in Fig. \ref{['Fig:ImagePhoto']}. The bold arrows indicate immediate links and the dashed arrows indicate ancestral links.
• Figure 4: Outline of the objective in this work. Given a set of near-duplicate face images belonging to the same subject, our objective is two-fold. Firstly, we use locally-scaled spectral clustering to identify each cluster (each cluster represents an IPT). The clusters indicated by ellipsoids vary in diameter indicating the importance of local scaling. Secondly, we reconstruct each IPT using graph theoretic node embedding and link prediction. The ensemble of IPTs results in an Image Phylogeny Forest (IPF).
• Figure 5: Outline of the two-step method used in this work. In the first step, the node embedding module, i.e., a graph neural network, $f\bm{(X,A)}$ accepts $\bm{X}$: features from each image in the IPT; features can be pixel intensity or convolutional features, and $\bm{A}$: an initial adjacency matrix as a pair of inputs. During training, the initial adjacency matrix denotes the ground truth relationship between the images in the IPT, while during testing, the initial adjacency matrix is populated by comparing distance between images features against a threshold to determine presence of edge $(=1)$ or absence of edge $(=0)$. The node embedding module produces a vector of depth labels corresponding to each IPT configuration. In the second step, the link prediction module accepts $g\bm{(l,N)}$, $\bm{l}$: depth labels from the first module, and $\bm{N}$: sensor pattern noise (PRNU) features computed from the near-duplicates as inputs. The link prediction module produces the reconstructed image phylogeny tree (IPT).