Comparative Analysis of Pre-trained Deep Learning Models and DINOv2 for Cushing's Syndrome Diagnosis in Facial Analysis

TL;DR

Cushing's syndrome diagnosis from facial images is challenging due to reliance on global facial cues. The study benchmarks pre-trained CNNs and Transformer-based models (ViT, Swin) against the DINOv2 foundation model on a clinical facial-image dataset, while also evaluating the impact of freezing the DINOv2 backbone and potential gender bias. Transformers and DINOv2 outperform CNN baselines, with ViT achieving the top F1 score around 85.7%; freezing DINOv2 backbones significantly improves performance and generalization. The results support transformer and foundation-model approaches for automated CS screening from facial data, but reveal gender bias likely driven by data imbalance and underscore the need for larger, multi-center, balanced datasets and possibly multi-view or 3D imaging to improve generalizability.

Abstract

Cushing's syndrome is a condition caused by excessive glucocorticoid secretion from the adrenal cortex, often manifesting with moon facies and plethora, making facial data crucial for diagnosis. Previous studies have used pre-trained convolutional neural networks (CNNs) for diagnosing Cushing's syndrome using frontal facial images. However, CNNs are better at capturing local features, while Cushing's syndrome often presents with global facial features. Transformer-based models like ViT and SWIN, which utilize self-attention mechanisms, can better capture long-range dependencies and global features. Recently, DINOv2, a foundation model based on visual Transformers, has gained interest. This study compares the performance of various pre-trained models, including CNNs, Transformer-based models, and DINOv2, in diagnosing Cushing's syndrome. We also analyze gender bias and the impact of freezing mechanisms on DINOv2. Our results show that Transformer-based models and DINOv2 outperformed CNNs, with ViT achieving the highest F1 score of 85.74%. Both the pre-trained model and DINOv2 had higher accuracy for female samples. DINOv2 also showed improved performance when freezing parameters. In conclusion, Transformer-based models and DINOv2 are effective for Cushing's syndrome classification.

Figures (2)

• Figure 1: Flow diagram of this study.
• Figure 2: Activation maps of the pre-trained model and the foundational model.