Denoising and Alignment: Rethinking Domain Generalization for Multimodal Face Anti-Spoofing

TL;DR

MMDA tackles multimodal face anti-spoofing generalization by leveraging a CLIP-based generalized representation space and a denoising-alignment pipeline. The Modality-Domain Joint Differential Attention (MD2A) suppresses domain and modality noise, while the Representation Space Soft (RS2) alignment preserves rich cross-modal representations; the U-shaped Dual Space Adaptation (U-DSA) enhances cross-layer generalization by remapping deep features back to a shared space. Across four benchmark datasets and multiple protocols, MMDA achieves state-of-the-art cross-domain generalization and robust multimodal detection, including scenarios with missing modalities. This approach promises improved security for multimodal FR systems operating under diverse real-world conditions by combining noise-robust representation learning with flexible cross-domain alignment.

Abstract

Face Anti-Spoofing (FAS) is essential for the security of facial recognition systems in diverse scenarios such as payment processing and surveillance. Current multimodal FAS methods often struggle with effective generalization, mainly due to modality-specific biases and domain shifts. To address these challenges, we introduce the \textbf{M}ulti\textbf{m}odal \textbf{D}enoising and \textbf{A}lignment (\textbf{MMDA}) framework. By leveraging the zero-shot generalization capability of CLIP, the MMDA framework effectively suppresses noise in multimodal data through denoising and alignment mechanisms, thereby significantly enhancing the generalization performance of cross-modal alignment. The \textbf{M}odality-\textbf{D}omain Joint \textbf{D}ifferential \textbf{A}ttention (\textbf{MD2A}) module in MMDA concurrently mitigates the impacts of domain and modality noise by refining the attention mechanism based on extracted common noise features. Furthermore, the \textbf{R}epresentation \textbf{S}pace \textbf{S}oft (\textbf{RS2}) Alignment strategy utilizes the pre-trained CLIP model to align multi-domain multimodal data into a generalized representation space in a flexible manner, preserving intricate representations and enhancing the model's adaptability to various unseen conditions. We also design a \textbf{U}-shaped \textbf{D}ual \textbf{S}pace \textbf{A}daptation (\textbf{U-DSA}) module to enhance the adaptability of representations while maintaining generalization performance. These improvements not only enhance the framework's generalization capabilities but also boost its ability to represent complex representations. Our experimental results on four benchmark datasets under different evaluation protocols demonstrate that the MMDA framework outperforms existing state-of-the-art methods in terms of cross-domain generalization and multimodal detection accuracy. The code will be released soon.

Figures (7)

• Figure 1: (a) In the single-modal FAS scenario, the presence of domain shifts leads to domain generalization issues. (b) In the multi-modal FAS scenario, the existence of modality biases causes the gap between the infrared and depth modalities to be significantly larger than that between RGB modalities. The combined effect of modality biases and domain shifts amplifies noise, making multi-modal FAS more challenging. (c) Our proposed method not only reduces noise but also avoids overly smooth decision boundaries, thereby alleviating the issue of test samples with severe domain shifts failing to be correctly distinguished.
• Figure 2: Overall framework of the proposed MMDA. (a) Overall process of MMDA. (b) Details of the U-shaped Dual Space Adaptation (U-DSA) module and the application method of the Representation Space Soft (RS2) alignment approach. (c) Operational details of the Modality-Domain Joint Differential Attention (MD2A).
• Figure 3: AUC statistics of the U-DSA Module across various caption groups at different depths. The height of each bar represents the number of captions achieving the specified AUC metric. Specifically, this analysis was conducted using a total of ten distinct caption sets to elucidate the impact and distribution of performance metrics at varying depths. This study provides insights into the behavior of the U-DSA at different depth levels and offers valuable perspectives for model optimization.
• Figure 4: A visualization of the statistics of the layers achieving the best alignment effects in different representation spaces constructed by U-DSA under various total layer numbers (1 to 7 layers).
• Figure 5: t-SNE visualization of the fine-tuned CLIP (left) and the classifier part of MMDA (right).