Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Intuition to Investigation: A Tool-Augmented Reasoning MLLM Framework for Generalizable Face Anti-Spoofing

Haoyuan Zhang, Keyao Wang, Guosheng Zhang, Haixiao Yue, Zhiwen Tan, Siran Peng, Tianshuo Zhang, Xiao Tan, Kunbin Chen, Wei He, Jingdong Wang, Ajian Liu, Xiangyu Zhu, Zhen Lei

TL;DR

This work proposes the Tool-Augmented Reasoning FAS (TAR-FAS) framework, which reformulates the FAS task as a Chain-of-Thought with Visual Tools (CoT-VT) paradigm, allowing MLLMs to begin with intuitive observations and adaptively invoke external visual tools for fine-grained investigation.

Abstract

Face recognition remains vulnerable to presentation attacks, calling for robust Face Anti-Spoofing (FAS) solutions. Recent MLLM-based FAS methods reformulate the binary classification task as the generation of brief textual descriptions to improve cross-domain generalization. However, their generalizability is still limited, as such descriptions mainly capture intuitive semantic cues (e.g., mask contours) while struggling to perceive fine-grained visual patterns. To address this limitation, we incorporate external visual tools into MLLMs to encourage deeper investigation of subtle spoof clues. Specifically, we propose the Tool-Augmented Reasoning FAS (TAR-FAS) framework, which reformulates the FAS task as a Chain-of-Thought with Visual Tools (CoT-VT) paradigm, allowing MLLMs to begin with intuitive observations and adaptively invoke external visual tools for fine-grained investigation. To this end, we design a tool-augmented data annotation pipeline and construct the ToolFAS-16K dataset, which contains multi-turn tool-use reasoning trajectories. Furthermore, we introduce a tool-aware FAS training pipeline, where Diverse-Tool Group Relative Policy Optimization (DT-GRPO) enables the model to autonomously learn efficient tool use. Extensive experiments under a challenging one-to-eleven cross-domain protocol demonstrate that TAR-FAS achieves SOTA performance while providing fine-grained visual investigation for trustworthy spoof detection.

From Intuition to Investigation: A Tool-Augmented Reasoning MLLM Framework for Generalizable Face Anti-Spoofing

TL;DR

This work proposes the Tool-Augmented Reasoning FAS (TAR-FAS) framework, which reformulates the FAS task as a Chain-of-Thought with Visual Tools (CoT-VT) paradigm, allowing MLLMs to begin with intuitive observations and adaptively invoke external visual tools for fine-grained investigation.

Abstract

Face recognition remains vulnerable to presentation attacks, calling for robust Face Anti-Spoofing (FAS) solutions. Recent MLLM-based FAS methods reformulate the binary classification task as the generation of brief textual descriptions to improve cross-domain generalization. However, their generalizability is still limited, as such descriptions mainly capture intuitive semantic cues (e.g., mask contours) while struggling to perceive fine-grained visual patterns. To address this limitation, we incorporate external visual tools into MLLMs to encourage deeper investigation of subtle spoof clues. Specifically, we propose the Tool-Augmented Reasoning FAS (TAR-FAS) framework, which reformulates the FAS task as a Chain-of-Thought with Visual Tools (CoT-VT) paradigm, allowing MLLMs to begin with intuitive observations and adaptively invoke external visual tools for fine-grained investigation. To this end, we design a tool-augmented data annotation pipeline and construct the ToolFAS-16K dataset, which contains multi-turn tool-use reasoning trajectories. Furthermore, we introduce a tool-aware FAS training pipeline, where Diverse-Tool Group Relative Policy Optimization (DT-GRPO) enables the model to autonomously learn efficient tool use. Extensive experiments under a challenging one-to-eleven cross-domain protocol demonstrate that TAR-FAS achieves SOTA performance while providing fine-grained visual investigation for trustworthy spoof detection.
Paper Structure (37 sections, 17 equations, 22 figures, 7 tables)

This paper contains 37 sections, 17 equations, 22 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Face Anti-Spoofing
  4. Multimodal Large Language Models
  5. Tool-Use MLLM Agent
  6. Methodology
  7. Tool-Augmented Data Annotation
  8. Data and Tool Selection
  9. Annotation Workflow
  10. Data Verification and Re-Annotation
  11. Tool-Aware FAS Training Pipeline
  12. FAS Knowledge Transfer
  13. Tool-call Format Injection
  14. Diverse-Tool GRPO
  15. Experiments
  16. ...and 22 more sections

Figures (22)

  • Figure 1: Previous methods struggle to distinguish high-quality spoof samples with subtle visual cues, while TAR-FAS enhances fine-grained investigation by coupling MLLM with visual tools.
  • Figure 2: Detailed construction and samples of ToolFAS-16K.
  • Figure 3: Illustration of the data annotation pipeline. (a) Each annotation can be divided into multiple sub-annotations $Ann^{(l)}$ generated by the MLLM with historical context, and the multi-turn process ends when the MLLM outputs the final answer. (b) The data verification process ensuring annotation reliability.
  • Figure 4: The total framework and training pipeline of TAR-FAS. (a) introduce the framework of TAR-FAS which can give a quick intuitive answer and investigate with visual tools to give a more accurate final decision. (b) illustrate the whole tool-aware FAS training pipeline including FAS knowledge transfer, tool-call format injection and DT-GRPO.
  • Figure 5: The visualization of TAR-FAS results. We select one sample for each type: real, print, replay, and mask attacks. The zoom in operation in red boxes is for better understandings.
  • ...and 17 more figures