Beyond Specialization: Assessing the Capabilities of MLLMs in Age and Gender Estimation

TL;DR

This study evaluates the capabilities of multimodal large language models (MLLMs) for age and gender estimation by comparing leading generalist models (ChatGPT-4V/4O, ShareGPT4V, LLaVA variants) against an enhanced specialized model, MiVOLO_384. It extends MiVOLO with larger input resolution and a bigger, more diverse training set ($LAGENDA_{ext}$), and introduces a robust evaluation across multiple benchmarks (IMDB-clean, LAGENDA, CACD, FairFace, Adience, NanoLAGENDA) including both face and body information. The work also explores fine-tuning MLLMs (notably ShareGPT4V and LLaVA variants) on the task, detailing training procedures, hyperparameters, and computational costs, and reports results that in many cases approach or surpass specialized models, with ChatGPT-4O achieving strong age estimates at the cost of higher resources and compliance constraints. Overall, the findings suggest that, while specialized models like MiVOLO_384 still offer top accuracy for age/gender estimation, well-tuned MLLMs can match or exceed them in several contexts, highlighting the potential shift toward versatile, general-purpose vision-language models for attribute estimation tasks.

Abstract

Multimodal Large Language Models (MLLMs) have recently gained immense popularity. Powerful commercial models like ChatGPT-4V and Gemini, as well as open-source ones such as LLaVA, are essentially general-purpose models and are applied to solve a wide variety of tasks, including those in computer vision. These neural networks possess such strong general knowledge and reasoning abilities that they have proven capable of working even on tasks for which they were not specifically trained. We compared the capabilities of the most powerful MLLMs to date: ShareGPT4V, ChatGPT, LLaVA-Next in a specialized task of age and gender estimation with our state-of-the-art specialized model, MiVOLO. We also updated MiVOLO and provide details and new metrics in this article. This comparison has yielded some interesting results and insights about the strengths and weaknesses of the participating models. Furthermore, we attempted various ways to fine-tune the ShareGPT4V model for this specific task, aiming to achieve state-of-the-art results in this particular challenge. Although such a model would not be practical in production, as it is incredibly expensive compared to a specialized model like MiVOLO, it could be very useful in some tasks, like data annotation.

Figures (4)

• Figure 1: An example of evaluated models predictions. The image illustrates output of specialized $MiVOLO_{384}$ model and different MLLMs. $MiVOLO_{384}$ makes predictions based on the face and body crops. Other models make predictions based on prompt and image of body crop.
• Figure 2: Graphical representation of the study focused on comparison specialized model for age and gender estimation and multimodal models.
• Figure 3: Relationship between MAE and age group across different models tested on the NanoLAGENDA benchmark.
• Figure 4: Failure case: Through manual analysis of ChatGPT's gender misclassification, we have discovered that it sometimes makes mistakes with examples where men have long hair. This is a humorously artificial illustrative example of such failure cases. Both MiVOLO and the fine-tuned as well as vanilla LLaVA models remain stable.