SegImgNet: Segmentation-Guided Dual-Branch Network for Retinal Disease Diagnoses

TL;DR

SegImgNet addresses the challenge of retinal disease diagnosis by fusing segmentation-derived structural features with global image context. It employs a segmentation module (U-Net) to produce multi-scale retinal structure maps and a dual-branch classifier built on ConvNeXt backbones, augmented by Segmentation-Guided Attention (SGA) to emphasize anatomical features. The model uses a Weighted Cross-Entropy loss to handle class imbalance and demonstrates superior sensitivity and overall performance compared to state-of-the-art baselines on both AIROGS and e-ROP datasets. These results suggest that incorporating retinal structural priors through segmentation maps can enhance robustness and clinical reliability in automated retinal analysis, with code made publicly available.

Abstract

Retinal image plays a crucial role in diagnosing various diseases, as retinal structures provide essential diagnostic information. However, effectively capturing structural features while integrating them with contextual information from retinal images remains a challenge. In this work, we propose segmentation-guided dual-branch network for retinal disease diagnosis using retinal images and their segmentation maps, named SegImgNet. SegImgNet incorporates a segmentation module to generate multi-scale retinal structural feature maps from retinal images. The classification module employs two encoders to independently extract features from segmented images and retinal images for disease classification. To further enhance feature extraction, we introduce the Segmentation-Guided Attention (SGA) block, which leverages feature maps from the segmentation module to refine the classification process. We evaluate SegImgNet on the public AIROGS dataset and the private e-ROP dataset. Experimental results demonstrate that SegImgNet consistently outperforms existing methods, underscoring its effectiveness in retinal disease diagnosis. The code is publicly available at https://github.com/hawk-sudo/SegImgNet.

Figures (3)

• Figure 1: Comparison of different approaches: a) Direct classification of raw images using a standard deep learning model. b) Classification based on the segmented image extracted from a segmentation model. c) Combined classification using both raw and segmented images in a shared encoder. d) Classification using both raw images and enhanced segmentation maps in dual encoders (ours).
• Figure 2: Overview of SegImgNet. The input retinal image is first processed by the segmentation module to generate segmentation maps (top). These segmentation maps, along with the raw retinal image, are then fed into separate encoders to extract disease-related features (bottom). The SGA block leverages intermediate feature maps from the segmentation module to generate attention maps, guiding the segmented image encoder's focus on retinal structural features. Finally, the classifier integrates outputs from both encoders for disease classification.
• Figure 3: Intermediate feature map visualizations of top-3 methods. (a) and (b) are from the AIROGS dataset, where (a) is healthy and (b) is glaucomatous. (c) and (d) are from the e-ROP dataset, where (c) is healthy and (d) is ROP.