Integrating Deep Learning with Fundus and Optical Coherence Tomography for Cardiovascular Disease Prediction

TL;DR

A novel binary classification network based on a Multi-channel Variational Autoencoder (MCVAE), which learns a latent embedding of patients' fundus and OCT images to classify individuals into two groups: those likely to develop CVD in the future and those who are not.

Abstract

Early identification of patients at risk of cardiovascular diseases (CVD) is crucial for effective preventive care, reducing healthcare burden, and improving patients' quality of life. This study demonstrates the potential of retinal optical coherence tomography (OCT) imaging combined with fundus photographs for identifying future adverse cardiac events. We used data from 977 patients who experienced CVD within a 5-year interval post-image acquisition, alongside 1,877 control participants without CVD, totaling 2,854 subjects. We propose a novel binary classification network based on a Multi-channel Variational Autoencoder (MCVAE), which learns a latent embedding of patients' fundus and OCT images to classify individuals into two groups: those likely to develop CVD in the future and those who are not. Our model, trained on both imaging modalities, achieved promising results (AUROC 0.78 +/- 0.02, accuracy 0.68 +/- 0.002, precision 0.74 +/- 0.02, sensitivity 0.73 +/- 0.02, and specificity 0.68 +/- 0.01), demonstrating its efficacy in identifying patients at risk of future CVD events based on their retinal images. This study highlights the potential of retinal OCT imaging and fundus photographs as cost-effective, non-invasive alternatives for predicting cardiovascular disease risk. The widespread availability of these imaging techniques in optometry practices and hospitals further enhances their potential for large-scale CVD risk screening. Our findings contribute to the development of standardized, accessible methods for early CVD risk identification, potentially improving preventive care strategies and patient outcomes.

Figures (6)

• Figure 1: The illustration shows the architecture of our model. During the pre-training phase (left), fundus photographs and OCT images are processed through a 2D CNN and a 3D CNN encoder-decoder network in a multi-channel VAE configuration, respectively. In the fine-tuning phase (right), the images are processed through the pre-trained encoder-decoder networks. The resulting latent vectors are aggregated and input into a transformer architecture, followed by a fully connected layer. The model undergoes end-to-end training in both phases.
• Figure 2: The bar chart displays the classification metrics for Fundus, OCT, and combined Fundus-OCT modalities. The metrics shown include Accuracy, Precision, Sensitivity, Specificity, and AUC (Area Under the Curve) values, with their respective standard deviations. The colors represent different modalities: Fundus (black), OCT (red), and Fundus-OCT (yellow). Each bar is labeled with its corresponding value.
• Figure 3: a) The top row displays a sequence of Optical Coherence Tomography (OCT) images, highlighting the different layers of the retina with yellow markers indicating regions of interest. b) The bottom row shows corresponding fundus images with vascular structures and regions of interest also marked in yellow. These images are utilized for analyzing and predicting cardiovascular disease risks based on retinal biomarkers.
• Figure 4: STROBE flow chart outlines the patient selection process for a study using the UK Biobank dataset with (Optical Coherence Tomography) OCT and Fundus images of the left eye.
• Figure 5: Heatmaps of Patients with Different Cardiovascular Diseases (CVDs)