Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Benchmarking Ophthalmology Foundation Models for Clinically Significant Age Macular Degeneration Detection

Benjamin A. Cohen, Jonathan Fhima, Meishar Meisel, Baskin Meital, Luis Filipe Nakayama, Eran Berkowitz, Joachim A. Behar

TL;DR

The results show that iBOT pretrained on natural images achieves the highest out-of-distribution generalization, outperforming domain-specific models, and challenge the assumption that in-domain pretraining is necessary.

Abstract

Self-supervised learning (SSL) has enabled Vision Transformers (ViTs) to learn robust representations from large-scale natural image datasets, enhancing their generalization across domains. In retinal imaging, foundation models pretrained on either natural or ophthalmic data have shown promise, but the benefits of in-domain pretraining remain uncertain. To investigate this, we benchmark six SSL-pretrained ViTs on seven digital fundus image (DFI) datasets totaling 70,000 expert-annotated images for the task of moderate-to-late age-related macular degeneration (AMD) identification. Our results show that iBOT pretrained on natural images achieves the highest out-of-distribution generalization, with AUROCs of 0.80-0.97, outperforming domain-specific models, which achieved AUROCs of 0.78-0.96 and a baseline ViT-L with no pretraining, which achieved AUROCs of 0.68-0.91. These findings highlight the value of foundation models in improving AMD identification and challenge the assumption that in-domain pretraining is necessary. Furthermore, we release BRAMD, an open-access dataset (n=587) of DFIs with AMD labels from Brazil.

Benchmarking Ophthalmology Foundation Models for Clinically Significant Age Macular Degeneration Detection

TL;DR

The results show that iBOT pretrained on natural images achieves the highest out-of-distribution generalization, outperforming domain-specific models, and challenge the assumption that in-domain pretraining is necessary.

Abstract

Self-supervised learning (SSL) has enabled Vision Transformers (ViTs) to learn robust representations from large-scale natural image datasets, enhancing their generalization across domains. In retinal imaging, foundation models pretrained on either natural or ophthalmic data have shown promise, but the benefits of in-domain pretraining remain uncertain. To investigate this, we benchmark six SSL-pretrained ViTs on seven digital fundus image (DFI) datasets totaling 70,000 expert-annotated images for the task of moderate-to-late age-related macular degeneration (AMD) identification. Our results show that iBOT pretrained on natural images achieves the highest out-of-distribution generalization, with AUROCs of 0.80-0.97, outperforming domain-specific models, which achieved AUROCs of 0.78-0.96 and a baseline ViT-L with no pretraining, which achieved AUROCs of 0.68-0.91. These findings highlight the value of foundation models in improving AMD identification and challenge the assumption that in-domain pretraining is necessary. Furthermore, we release BRAMD, an open-access dataset (n=587) of DFIs with AMD labels from Brazil.
Paper Structure (5 sections, 3 figures, 4 tables)

This paper contains 5 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Datasets
  3. Methods
  4. Results
  5. Acknowledgments

Figures (3)

  • Figure 1: Models performance for AMD identification. (a) Results for a ViT-L model (no pretraining) as well as a set of six foundation models fine-tuned on AREDS-train. The performance is presented as the average AUROC over the AREDS-test and the six external datasets (target domains). (b) The best backbone identified, namely iBOT, is fine-tuned using an all-against-one multi-source domain training approach. For the resulting model, denoted AMDNet, OOD performance is reported for each domain. AMDNet performance is also compared to a state-of-the-art AMD detection model called DeepSeeNet deepseenet2019.
  • Figure 2: Error analysis. a) AMDNet probability output for the different subgroups for AREDS; b) AUROC per age group for BRAMD. C: number of control images, A: number of AMD images in the age bin; c) False positive rate per comorbidity for RFMiD1. LD: large drusen; GA: geographic atrophy; NVAMD: Neovascular AMD.
  • Figure 3: Examples of attention maps of AMDNet on images from BRAMD. We used GradCAM gradcam to plot in red the regions used by the model to generate its predictions. $p$ is the probability for AMD given by AMDNet.