Benchmarking Multimodal Large Language Models for Face Recognition
Hatef Otroshi Shahreza, Sébastien Marcel
TL;DR
This work tackles the problem of evaluating open-source multimodal large language models (MLLMs) on face recognition, a task traditionally dominated by specialized models. It introduces a verification-based benchmark across standard datasets (LFW, CALFW, CPLFW, CFP, AgeDB, RFW) with a fixed 6,000-pair per-dataset protocol and uses a simple yes/no prompt to assess identity similarity, benchmarking many MLLMs against AdaFace/IResNet-50 baselines. Key contributions include a comprehensive, public benchmark framework, evidence that MLLMs can capture semantic cues but struggle with high-precision identity verification, and the demonstration that domain-specific fine-tuning (as in FaceLLM) can improve performance. The results highlight a persistent gap between open-source MLLMs and specialized face-recognition systems, particularly in cross-demographic scenarios, and suggest directions for model design and data curation to enhance accuracy and generalization. The publicly available source code provides a foundation for fair comparisons and future progress in MLLM-based face recognition.
Abstract
Multimodal large language models (MLLMs) have achieved remarkable performance across diverse vision-and-language tasks. However, their potential in face recognition remains underexplored. In particular, the performance of open-source MLLMs needs to be evaluated and compared with existing face recognition models on standard benchmarks with similar protocol. In this work, we present a systematic benchmark of state-of-the-art MLLMs for face recognition on several face recognition datasets, including LFW, CALFW, CPLFW, CFP, AgeDB and RFW. Experimental results reveal that while MLLMs capture rich semantic cues useful for face-related tasks, they lag behind specialized models in high-precision recognition scenarios in zero-shot applications. This benchmark provides a foundation for advancing MLLM-based face recognition, offering insights for the design of next-generation models with higher accuracy and generalization. The source code of our benchmark is publicly available in the project page.