Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multimodal Models Meet Presentation Attack Detection on ID Documents

Marina Villanueva, Juan M. Espin, Juan E. Tapia

Abstract

The integration of multimodal models into Presentation Attack Detection (PAD) for ID Documents represents a significant advancement in biometric security. Traditional PAD systems rely solely on visual features, which often fail to detect sophisticated spoofing attacks. This study explores the combination of visual and textual modalities by utilizing pre-trained multimodal models, such as Paligemma, Llava, and Qwen, to enhance the detection of presentation attacks on ID Documents. This approach merges deep visual embeddings with contextual metadata (e.g., document type, issuer, and date). However, experimental results indicate that these models struggle to accurately detect PAD on ID Documents.

Multimodal Models Meet Presentation Attack Detection on ID Documents

Abstract

The integration of multimodal models into Presentation Attack Detection (PAD) for ID Documents represents a significant advancement in biometric security. Traditional PAD systems rely solely on visual features, which often fail to detect sophisticated spoofing attacks. This study explores the combination of visual and textual modalities by utilizing pre-trained multimodal models, such as Paligemma, Llava, and Qwen, to enhance the detection of presentation attacks on ID Documents. This approach merges deep visual embeddings with contextual metadata (e.g., document type, issuer, and date). However, experimental results indicate that these models struggle to accurately detect PAD on ID Documents.

Paper Structure

This paper contains 10 sections, 2 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Metrics
  5. Dataset
  6. Experiment and Results
  7. Experiment 1. Definition Accuracy
  8. Experiment 2. Visual Description
  9. Experiment 3. Prompt Design Strategy
  10. Conclusions

Figures (6)

  • Figure 1: Multi-turn strategy.
  • Figure 2: Scoring process based on model output logits applied to each model. Example for prompts with answer "Yes" or "No".
  • Figure 3: Attack examples for ID Cards and Passports.
  • Figure 4: Results of Exp. 1. Comparison of the use of technical terminology among models.
  • Figure 5: Visual keypoints of a spoof PVC ID Card.
  • ...and 1 more figures