Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MADation: Face Morphing Attack Detection with Foundation Models

Eduarda Caldeira, Guray Ozgur, Tahar Chettaoui, Marija Ivanovska, Peter Peer, Fadi Boutros, Vitomir Struc, Naser Damer

TL;DR

MADation introduces the first adaptation of foundation models to Morphing Attack Detection by fine-tuning CLIP with rsLoRA while training a binary classifier. The method leverages CLIP’s zero-shot generalization and adds a targeted domain alignment to the MAD task, achieving superior or competitive results against both FM-based and transformer baselines. Across SMDD, MAD22, MorDIFF, and FRLL-Morphs benchmarks, MADation demonstrates robust performance gains and highlights the value of FM adaptation in biometrics with limited labeled data. The work provides reproducible code to foster further research in MAD and foundation-model applications in biometrics.

Abstract

Despite the considerable performance improvements of face recognition algorithms in recent years, the same scientific advances responsible for this progress can also be used to create efficient ways to attack them, posing a threat to their secure deployment. Morphing attack detection (MAD) systems aim to detect a specific type of threat, morphing attacks, at an early stage, preventing them from being considered for verification in critical processes. Foundation models (FM) learn from extensive amounts of unlabelled data, achieving remarkable zero-shot generalization to unseen domains. Although this generalization capacity might be weak when dealing with domain-specific downstream tasks such as MAD, FMs can easily adapt to these settings while retaining the built-in knowledge acquired during pre-training. In this work, we recognize the potential of FMs to perform well in the MAD task when properly adapted to its specificities. To this end, we adapt FM CLIP architectures with LoRA weights while simultaneously training a classification header. The proposed framework, MADation surpasses our alternative FM and transformer-based frameworks and constitutes the first adaption of FMs to the MAD task. MADation presents competitive results with current MAD solutions in the literature and even surpasses them in several evaluation scenarios. To encourage reproducibility and facilitate further research in MAD, we publicly release the implementation of MADation at https://github.com/gurayozgur/MADation

MADation: Face Morphing Attack Detection with Foundation Models

TL;DR

MADation introduces the first adaptation of foundation models to Morphing Attack Detection by fine-tuning CLIP with rsLoRA while training a binary classifier. The method leverages CLIP’s zero-shot generalization and adds a targeted domain alignment to the MAD task, achieving superior or competitive results against both FM-based and transformer baselines. Across SMDD, MAD22, MorDIFF, and FRLL-Morphs benchmarks, MADation demonstrates robust performance gains and highlights the value of FM adaptation in biometrics with limited labeled data. The work provides reproducible code to foster further research in MAD and foundation-model applications in biometrics.

Abstract

Despite the considerable performance improvements of face recognition algorithms in recent years, the same scientific advances responsible for this progress can also be used to create efficient ways to attack them, posing a threat to their secure deployment. Morphing attack detection (MAD) systems aim to detect a specific type of threat, morphing attacks, at an early stage, preventing them from being considered for verification in critical processes. Foundation models (FM) learn from extensive amounts of unlabelled data, achieving remarkable zero-shot generalization to unseen domains. Although this generalization capacity might be weak when dealing with domain-specific downstream tasks such as MAD, FMs can easily adapt to these settings while retaining the built-in knowledge acquired during pre-training. In this work, we recognize the potential of FMs to perform well in the MAD task when properly adapted to its specificities. To this end, we adapt FM CLIP architectures with LoRA weights while simultaneously training a classification header. The proposed framework, MADation surpasses our alternative FM and transformer-based frameworks and constitutes the first adaption of FMs to the MAD task. MADation presents competitive results with current MAD solutions in the literature and even surpasses them in several evaluation scenarios. To encourage reproducibility and facilitate further research in MAD, we publicly release the implementation of MADation at https://github.com/gurayozgur/MADation
Paper Structure (9 sections, 5 equations, 1 figure, 3 tables)

This paper contains 9 sections, 5 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Preliminary on CLIP
  5. MADation
  6. MAD Baselines
  7. Experimental Setup
  8. Results and Discussion
  9. Conclusion

Figures (1)

  • Figure 1: Morphing attack generation and MADation's pipeline. The left side of the figure depicts a morphing sample and the two bona-fide identities that were morphed to generate it, using damer2023mordiff. Keep in mind that attackers commonly choose to morph faces with similar features for higher success DBLP:conf/icb/DamerSZWTKK19. The right side represents MADation's pipeline, consisting of an adapted FM followed by a binary fully connected classification layer. The embedding space of the FM is adapted by fine-tuning the LoRA parameters and the classification layer is simultaneously trained to produce the MAD predictions. Better visualized in colour.