Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Temporal-Enhanced Interpretable Multi-Modal Prognosis and Risk Stratification Framework for Diabetic Retinopathy (TIMM-ProRS)

Susmita Kar, A S M Ahsanul Sarkar Akib, Abdul Hasib, Samin Yaser, Anas Bin Azim

TL;DR

Diabetic retinopathy poses rising global vision-loss risk and diagnostic ambiguity, especially in underserved regions. The paper introduces TIMM-ProRS, a multi-modal framework that fuses CNN-derived local retinal features, ViT-based global context, and GNN-modeled temporal biomarkers to produce DR grading and 5-year progression risk, augmented with interpretability and uncertainty quantification. The approach achieves state-of-the-art results across diverse datasets (APTOS 2019, Messidor-2, EyePACS, Messidor-1, RFMiD), including $97.8%$ accuracy and a $0.89$ C-index for progression risk, while providing saliency maps with clinician-aligned explanations and robust generalization. This work advances telemedicine readiness by delivering early, interpretable prognosis and scalable risk stratification, with future plans for real-world longitudinal validation and expanded biomarker integration (e.g., VEGF) to further personalize care.

Abstract

Diabetic retinopathy (DR), affecting millions globally with projections indicating a significant rise, poses a severe blindness risk and strains healthcare systems. Diagnostic complexity arises from visual symptom overlap with conditions like age-related macular degeneration and hypertensive retinopathy, exacerbated by high misdiagnosis rates in underserved regions. This study introduces TIMM-ProRS, a novel deep learning framework integrating Vision Transformer (ViT), Convolutional Neural Network (CNN), and Graph Neural Network (GNN) with multi-modal fusion. TIMM-ProRS uniquely leverages both retinal images and temporal biomarkers (HbA1c, retinal thickness) to capture multi-modal and temporal dynamics. Evaluated comprehensively across diverse datasets including APTOS 2019 (trained), Messidor-2, RFMiD, EyePACS, and Messidor-1 (validated), the model achieves 97.8\% accuracy and an F1-score of 0.96, demonstrating state-of-the-art performance and outperforming existing methods like RSG-Net and DeepDR. This approach enables early, precise, and interpretable diagnosis, supporting scalable telemedical management and enhancing global eye health sustainability.

Temporal-Enhanced Interpretable Multi-Modal Prognosis and Risk Stratification Framework for Diabetic Retinopathy (TIMM-ProRS)

TL;DR

Diabetic retinopathy poses rising global vision-loss risk and diagnostic ambiguity, especially in underserved regions. The paper introduces TIMM-ProRS, a multi-modal framework that fuses CNN-derived local retinal features, ViT-based global context, and GNN-modeled temporal biomarkers to produce DR grading and 5-year progression risk, augmented with interpretability and uncertainty quantification. The approach achieves state-of-the-art results across diverse datasets (APTOS 2019, Messidor-2, EyePACS, Messidor-1, RFMiD), including accuracy and a C-index for progression risk, while providing saliency maps with clinician-aligned explanations and robust generalization. This work advances telemedicine readiness by delivering early, interpretable prognosis and scalable risk stratification, with future plans for real-world longitudinal validation and expanded biomarker integration (e.g., VEGF) to further personalize care.

Abstract

Diabetic retinopathy (DR), affecting millions globally with projections indicating a significant rise, poses a severe blindness risk and strains healthcare systems. Diagnostic complexity arises from visual symptom overlap with conditions like age-related macular degeneration and hypertensive retinopathy, exacerbated by high misdiagnosis rates in underserved regions. This study introduces TIMM-ProRS, a novel deep learning framework integrating Vision Transformer (ViT), Convolutional Neural Network (CNN), and Graph Neural Network (GNN) with multi-modal fusion. TIMM-ProRS uniquely leverages both retinal images and temporal biomarkers (HbA1c, retinal thickness) to capture multi-modal and temporal dynamics. Evaluated comprehensively across diverse datasets including APTOS 2019 (trained), Messidor-2, RFMiD, EyePACS, and Messidor-1 (validated), the model achieves 97.8\% accuracy and an F1-score of 0.96, demonstrating state-of-the-art performance and outperforming existing methods like RSG-Net and DeepDR. This approach enables early, precise, and interpretable diagnosis, supporting scalable telemedical management and enhancing global eye health sustainability.
Paper Structure (23 sections, 9 equations, 6 figures, 6 tables, 1 algorithm)

This paper contains 23 sections, 9 equations, 6 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Methodology
  4. System Architecture
  5. Local Feature Extraction
  6. Global Context Modeling
  7. Temporal Biomarker Processing
  8. Multi-Modal Fusion
  9. Interpretability & Uncertainty
  10. Data
  11. Preprocessing
  12. Training
  13. Evaluation
  14. Results and Analysis
  15. Quantitative Performance
  16. ...and 8 more sections

Figures (6)

  • Figure 1: TIMM-ProRS architecture integrating CNN, ViT, GNN, and multi-modal fusion with uncertainty quantification.
  • Figure 2: Preprocessing: (a) Original, (b) CLAHE-enhanced, (c) Denoised, (d) Resized.
  • Figure 3: ROC Curves for TIMM-ProRS Across Datasets
  • Figure 4: Confusion Matrix for TIMM-ProRS on APTOS 2019
  • Figure 5: Decision Curve Analysis (DCA) Plot for TIMM-ProRS
  • ...and 1 more figures