Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring the Transferability of a Foundation Model for Fundus Images: Application to Hypertensive Retinopathy

Julio Silva-Rodriguez, Jihed Chelbi, Waziha Kabir, Hadi Chakor, Jose Dolz, Ismail Ben Ayed, Riadh Kobbi

TL;DR

This study assesses whether the retina-focused foundation model FLAIR can transfer to hypertensive retinopathy tasks in fundus images, versus traditional ImageNet pretraining. By evaluating Linear Probing and Fine-Tuning, and their LP+FT variant, on the CGI-HRDC dataset, the work finds that direct transfer via FLAIR LP provides modest gains (~2.5%), while full fine-tuning yields larger improvements, especially for Hypertensive Retinopathy detection. Importantly, initializing the classifier from FLAIR features and then finetuning (LP+FT) delivers the strongest and most consistent performance gains, outperforming Imagenet baselines in several configurations. The hidden-test results show competitive rankings but reveal a performance drop relative to cross-validation, underscoring distributional challenges and the potential of domain-specific foundation models for fundus analysis as a promising research direction.

Abstract

Using deep learning models pre-trained on Imagenet is the traditional solution for medical image classification to deal with data scarcity. Nevertheless, relevant literature supports that this strategy may offer limited gains due to the high dissimilarity between domains. Currently, the paradigm of adapting domain-specialized foundation models is proving to be a promising alternative. However, how to perform such knowledge transfer, and the benefits and limitations it presents, are under study. The CGI-HRDC challenge for Hypertensive Retinopathy diagnosis on fundus images introduces an appealing opportunity to evaluate the transferability of a recently released vision-language foundation model of the retina, FLAIR. In this work, we explore the potential of using FLAIR features as starting point for fundus image classification, and we compare its performance with regard to Imagenet initialization on two popular transfer learning methods: Linear Probing (LP) and Fine-Tuning (FP). Our empirical observations suggest that, in any case, the use of the traditional strategy provides performance gains. In contrast, direct transferability from FLAIR model allows gains of 2.5%. When fine-tuning the whole network, the performance gap increases up to 4%. In this case, we show that avoiding feature deterioration via LP initialization of the classifier allows the best re-use of the rich pre-trained features. Although direct transferability using LP still offers limited performance, we believe that foundation models such as FLAIR will drive the evolution of deep-learning-based fundus image analysis.

Exploring the Transferability of a Foundation Model for Fundus Images: Application to Hypertensive Retinopathy

TL;DR

This study assesses whether the retina-focused foundation model FLAIR can transfer to hypertensive retinopathy tasks in fundus images, versus traditional ImageNet pretraining. By evaluating Linear Probing and Fine-Tuning, and their LP+FT variant, on the CGI-HRDC dataset, the work finds that direct transfer via FLAIR LP provides modest gains (~2.5%), while full fine-tuning yields larger improvements, especially for Hypertensive Retinopathy detection. Importantly, initializing the classifier from FLAIR features and then finetuning (LP+FT) delivers the strongest and most consistent performance gains, outperforming Imagenet baselines in several configurations. The hidden-test results show competitive rankings but reveal a performance drop relative to cross-validation, underscoring distributional challenges and the potential of domain-specific foundation models for fundus analysis as a promising research direction.

Abstract

Using deep learning models pre-trained on Imagenet is the traditional solution for medical image classification to deal with data scarcity. Nevertheless, relevant literature supports that this strategy may offer limited gains due to the high dissimilarity between domains. Currently, the paradigm of adapting domain-specialized foundation models is proving to be a promising alternative. However, how to perform such knowledge transfer, and the benefits and limitations it presents, are under study. The CGI-HRDC challenge for Hypertensive Retinopathy diagnosis on fundus images introduces an appealing opportunity to evaluate the transferability of a recently released vision-language foundation model of the retina, FLAIR. In this work, we explore the potential of using FLAIR features as starting point for fundus image classification, and we compare its performance with regard to Imagenet initialization on two popular transfer learning methods: Linear Probing (LP) and Fine-Tuning (FP). Our empirical observations suggest that, in any case, the use of the traditional strategy provides performance gains. In contrast, direct transferability from FLAIR model allows gains of 2.5%. When fine-tuning the whole network, the performance gap increases up to 4%. In this case, we show that avoiding feature deterioration via LP initialization of the classifier allows the best re-use of the rich pre-trained features. Although direct transferability using LP still offers limited performance, we believe that foundation models such as FLAIR will drive the evolution of deep-learning-based fundus image analysis.
Paper Structure (15 sections, 1 figure, 2 tables)

This paper contains 15 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Transfer learning on fundus images
  4. FLAIR
  5. Transferability
  6. Method: Transfer Learning from FLAIR model
  7. Experiments
  8. Dataset
  9. Implementation details
  10. Baselines
  11. Evaluation protocol and metrics
  12. Results
  13. Development dataset results
  14. CGI-HRDC hidden test results
  15. Conclusions

Figures (1)

  • Figure 1: Standard vs. Foundation Model Paradigms. Deep learning solutions on medical image analysis are traditionally built upon models pre-trained on ImageNet to alleviate the need for large datasets. Nevertheless, the benefits of transfer learning might be limited when a substantial domain gap from source to target exists raghu2019transfusion. Foundation models on specific domains, such as FLAIR FLAIR for fundus image analysis, which is pre-trained on heterogeneous data sources and tasks, offer better resource-efficient transferability to new tasks.