IDRetracor: Towards Visual Forensics Against Malicious Face Swapping

TL;DR

This work tackles the lack of traceable evidence in deepfake detection by introducing face retracing, a visual-forensics task that recovers the original target face from a fake. It proposes IDRetracor, a network that uses mapping-aware convolutions guided by a Mapping Resolver to adapt to arbitrary target IDs and multiple face-swapping methods, ensuring retraced outputs lie in the target identity space. Trained on a purpose-built target-fake dataset with thousands of target IDs, the method achieves higher identity fidelity (ArcFace similarity > 0.65) and strong target-attribution performance across seen/unseen IDs and methods. The framework provides reliable forensic evidence and paves the way for practical accountability in face-swapping scenarios, though it requires large, source-specific datasets and acknowledges limitations when target attributes are severely degraded.

Abstract

The face swapping technique based on deepfake methods poses significant social risks to personal identity security. While numerous deepfake detection methods have been proposed as countermeasures against malicious face swapping, they can only output binary labels (Fake/Real) for distinguishing fake content without reliable and traceable evidence. To achieve visual forensics and target face attribution, we propose a novel task named face retracing, which considers retracing the original target face from the given fake one via inverse mapping. Toward this goal, we propose an IDRetracor that can retrace arbitrary original target identities from fake faces generated by multiple face swapping methods. Specifically, we first adopt a mapping resolver to perceive the possible solution space of the original target face for the inverse mappings. Then, we propose mapping-aware convolutions to retrace the original target face from the fake one. Such convolutions contain multiple kernels that can be combined under the control of the mapping resolver to tackle different face swapping mappings dynamically. Extensive experiments demonstrate that the IDRetracor exhibits promising retracing performance from both quantitative and qualitative perspectives.

Figures (16)

• Figure 1: The face retracing task for visual forensics and target face attribution. The deepfake detector can only provide a straightforward Fake/Real label without further information for subsequent actions against the perpetrators. In contrast, the retracor can visualize a retraced face with more traceability and reliability for forensics.
• Figure 2: Illustration on the significance of the two premises. The purple squares indicate the residual artifacts in $I^f$. The slimmer blue line denotes the mapping of IDRetracor for solving the original target face. Two thicker blue lines denote the constraint from the two premises for locating the possible solution space. Specifically, the lower line represents convergence towards the direction away from the source ID, while the higher one represents convergence towards the implicit target ID retained in the artifacts.
• Figure 3: The architecture of the proposed IDRetracor for retracing the original target faces. The mapping resolver can generate various weight vectors according to the input fake faces. These vectors then control the mapping-aware convolutions to recombine a kernel specifically for the input fake faces. Finally, the different recombined kernels process their corresponding fake faces and produce the final retraced faces.
• Figure 4: The applicability scope of the retracing task. The region encircled by a yellow line represents the ideal application scope, wherein neither the target nor the source identity is seen during training. The scope of the proposed method is indicated by the red-line region, where the source identity for testing is seen during the training phase, while the specific source face and the target identity are kept entirely unseen.
• Figure 5: Examples of source faces from VGGFace2 to craft our target-fake dataset.