FaceShield: Explainable Face Anti-Spoofing with Multimodal Large Language Models

TL;DR

FaceShield proposes an explainable face anti-spoofing framework built on a multitask multimodal large language model. By introducing Spoof-Aware Vision Perception (SAVP) and Prompt-Guided Vision Token Masking (PVTM), it extends FAS to four tasks—coarse-grained and fine-grained classification, reasoning, and attack localization—and leverages two dedicated datasets FaceShield-pre10K and FaceShield-sft45K for pretraining and supervised fine-tuning. Across rigorous intra- and cross-dataset evaluations, FaceShield consistently outperforms traditional FAS models and general MLLMs, while providing interpretable reasoning and precise localization of spoof regions. The work delivers a data-generation pipeline, a specialized FAS-focused MLLM, and comprehensive ablation analyses, with resources to be released to foster further progress in explainable FAS.

Abstract

Face anti-spoofing (FAS) is crucial for protecting facial recognition systems from presentation attacks. Previous methods approached this task as a classification problem, lacking interpretability and reasoning behind the predicted results. Recently, multimodal large language models (MLLMs) have shown strong capabilities in perception, reasoning, and decision-making in visual tasks. However, there is currently no universal and comprehensive MLLM and dataset specifically designed for FAS task. To address this gap, we propose FaceShield, a MLLM for FAS, along with the corresponding pre-training and supervised fine-tuning (SFT) datasets, FaceShield-pre10K and FaceShield-sft45K. FaceShield is capable of determining the authenticity of faces, identifying types of spoofing attacks, providing reasoning for its judgments, and detecting attack areas. Specifically, we employ spoof-aware vision perception (SAVP) that incorporates both the original image and auxiliary information based on prior knowledge. We then use an prompt-guided vision token masking (PVTM) strategy to random mask vision tokens, thereby improving the model's generalization ability. We conducted extensive experiments on three benchmark datasets, demonstrating that FaceShield significantly outperforms previous deep learning models and general MLLMs on four FAS tasks, i.e., coarse-grained classification, fine-grained classification, reasoning, and attack localization. Our instruction datasets, protocols, and codes will be released at https://github.com/Why0912/FaceShield.

Figures (24)

• Figure 1: FaceShield Multi-task Response Demonstration. This figure shows the model's performance on four tasks: coarse-grained classification (real vs. spoofed faces), fine-grained classification (specific attack types like print attacks), reasoning (explaining spoofing using features such as lighting and symmetry), and localization (detecting spoofed regions). It highlights FaceShield's ability to handle diverse, complex questions accurately.
• Figure 2: Pipelines of different FAS methods (a) traditional deep learning models, (b) multimodal models, and (c) MLLM
• Figure 3: Construction pipeline of our proposed instruction datasets (i.e., FaceShield-pre10K and FaceShield-sft45K).The initial datasets (WMCA, PADISI, SiW-Mv2) are combined to form a uni-class dataset covering 12 spoofing types, with selected images annotated for visual grounding. Using MLLM with structured prompts, we generate two datasets: a pretraining dataset and an SFT dataset divided into four tasks (coarse-grained classification, fine-grained classification, reasoning, and localization). The pretraining data is filtered by CLIP for similarity, producing the FaceShield-pre10k dataset. SFT data undergoes multi-level filtering (LLM-based, keywords, and human reviews), followed by augmentation, resulting in the FaceShield-sft45k dataset. Additional details can be found in the appendix.
• Figure 4: Proposed model architectures. (a) Proposed model with Spoof-Aware Vision Perception (SAVP). (b) Proposed model with SAVP and Prompt-Guided Vision Token Masking (PVTM). (c) Details about PVTM.
• Figure 5: Comparison of performance after fine-tuning using our proposed dataset on LLaVA and Bunny models.