Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Unknown Spoof Prompts for Generalized Face Anti-Spoofing Using Only Real Face Images

Fangling Jiang, Qi Li, Weining Wang, Wei Shen, Bing Liu, Zhenan Sun

TL;DR

The paper tackles the generalization gap in face anti-spoofing by addressing both covariate and semantic shifts. It introduces an unknown spoof prompt learning framework that derives real and spoof prompts from real-face data alone, leveraging a vision–language model (CLIP-like) to adapt pre-trained knowledge to unseen attacks. Four regularizers—contrastive generation, diversity refinement, prior knowledge guidance, and one-class discriminative classification—drive the learning of diverse, semantically meaningful prompts that distinguish real faces from potential unknown spoofs. Extensive experiments across nine datasets show state-of-the-art generalization to unseen attack types with no spoof-face training data and significant improvements in ACER and AUC across protocols, highlighting practical impact for low-cost, robust FAS deployment.

Abstract

Face anti-spoofing is a critical technology for ensuring the security of face recognition systems. However, its ability to generalize across diverse scenarios remains a significant challenge. In this paper, we attribute the limited generalization ability to two key factors: covariate shift, which arises from external data collection variations, and semantic shift, which results from substantial differences in emerging attack types. To address both challenges, we propose a novel approach for learning unknown spoof prompts, relying solely on real face images from a single source domain. Our method generates textual prompts for real faces and potential unknown spoof attacks by leveraging the general knowledge embedded in vision-language models, thereby enhancing the model's ability to generalize to unseen target domains. Specifically, we introduce a diverse spoof prompt optimization framework to learn effective prompts. This framework constrains unknown spoof prompts within a relaxed prior knowledge space while maximizing their distance from real face images. Moreover, it enforces semantic independence among different spoof prompts to capture a broad range of spoof patterns. Experimental results on nine datasets demonstrate that the learned prompts effectively transfer the knowledge of vision-language models, enabling state-of-the-art generalization ability against diverse unknown attack types across unseen target domains without using any spoof face images.

Learning Unknown Spoof Prompts for Generalized Face Anti-Spoofing Using Only Real Face Images

TL;DR

The paper tackles the generalization gap in face anti-spoofing by addressing both covariate and semantic shifts. It introduces an unknown spoof prompt learning framework that derives real and spoof prompts from real-face data alone, leveraging a vision–language model (CLIP-like) to adapt pre-trained knowledge to unseen attacks. Four regularizers—contrastive generation, diversity refinement, prior knowledge guidance, and one-class discriminative classification—drive the learning of diverse, semantically meaningful prompts that distinguish real faces from potential unknown spoofs. Extensive experiments across nine datasets show state-of-the-art generalization to unseen attack types with no spoof-face training data and significant improvements in ACER and AUC across protocols, highlighting practical impact for low-cost, robust FAS deployment.

Abstract

Face anti-spoofing is a critical technology for ensuring the security of face recognition systems. However, its ability to generalize across diverse scenarios remains a significant challenge. In this paper, we attribute the limited generalization ability to two key factors: covariate shift, which arises from external data collection variations, and semantic shift, which results from substantial differences in emerging attack types. To address both challenges, we propose a novel approach for learning unknown spoof prompts, relying solely on real face images from a single source domain. Our method generates textual prompts for real faces and potential unknown spoof attacks by leveraging the general knowledge embedded in vision-language models, thereby enhancing the model's ability to generalize to unseen target domains. Specifically, we introduce a diverse spoof prompt optimization framework to learn effective prompts. This framework constrains unknown spoof prompts within a relaxed prior knowledge space while maximizing their distance from real face images. Moreover, it enforces semantic independence among different spoof prompts to capture a broad range of spoof patterns. Experimental results on nine datasets demonstrate that the learned prompts effectively transfer the knowledge of vision-language models, enabling state-of-the-art generalization ability against diverse unknown attack types across unseen target domains without using any spoof face images.
Paper Structure (22 sections, 7 equations, 10 figures, 9 tables)

This paper contains 22 sections, 7 equations, 10 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Domain Generalization-based Face Anti-Spoofing
  4. One-Class Face Anti-Spoofing
  5. Proposed Method
  6. Problem Definition
  7. Spoof Prompt Contrastive Generation
  8. Spoof Prompt Diversity Refinement
  9. Prior Spoof Knowledge Guidance
  10. One-Class Discriminative Classification
  11. Overall Training Object and Inference
  12. Experiments
  13. Experimental Setups
  14. Comparison with State-of-the-Art Face Anti-Spoofing Methods
  15. Comparison with Classic Prompt Learning Methods
  16. ...and 7 more sections

Figures (10)

  • Figure 1: Conventional domain generalization (DG)-based FAS methods primarily focus on addressing covariate shift, but generalize poor to semantic shift caused by the emergence of unknown attack types. We learn potential spoof face prompts for unknown attack types solely based on real face images, adapting the pre-trained knowledge of the vision-language model to generalize well to semantic shift and covariate shift simultaneously present in unseen target domains.
  • Figure 2: Overview of the unknown spoof prompt learning framework for generalized face anti-spoofing. This framework is centered on real face training images, iteratively extrapolating within the embedding space to generate unknown spoof prompts for potential attack types. To ensure the effectiveness of the learned prompts, four modules are proposed: a spoof prompt contrastive generation module, which ensures that the distance between unknown spoof prompts and real face images exceeds that between real prompts and real face images; a prior spoof knowledge guidance module, which constrains unknown spoof prompts within the sparse range of prior spoof knowledge; a spoof prompt diversity refinement module, which enforces semantic independence among different unknown spoof prompts; a one-class discrimination module, which minimizes the discrepancy between the predicted probability distribution of real images and the ground-truth label distribution.
  • Figure 3: Sample images from the nine datasets. (a) Due to variations in recording equipment and environments, the real face images across the nine datasets exhibit significant covariate shift in their visual characteristics. (b) In addition to covariate shift, the diversity of attack types results in significant semantic shift due to variations in spoof patterns across different attack types.
  • Figure 4: Overall performance comparison across all the 27 protocols. (100-ACER)$\uparrow$(%) on protocols A1-A14 and B1-B7. (100-HTER)$\uparrow$(%) on on protocols P1-P6.
  • Figure 5: Challenging samples under the A14 obfuscation makeup protocol.
  • ...and 5 more figures