oculomix: Hierarchical Sampling for Retinal-Based Systemic Disease Prediction

TL;DR

The paper tackles the challenge of predicting systemic diseases from retinal images (oculomics) when standard mixed-sample data augmentations can perturb patient-specific attributes. It introduces Oculomix, a hierarchical sampling framework that confines mixing to exam-level (same time point) and patient-level (across time) groups, using a soft temporal ranking loss to encode disease progression across exams. On the AlzEye dataset for five-year MACE prediction and an external HYU MACE set, Oculomix yields consistent improvements in AUROC, AUPRC, and C-index (e.g., up to 3 percentage points in AUROC and 12% in C-index), demonstrating better preservation of patient morbidity signals and improved generalization. This approach provides a practically applicable augmentation strategy for transformer-based oculomics pipelines, highlighting the importance of incorporating patient-exam hierarchy and temporal information in mixed-sample augmentation.

Abstract

Oculomics - the concept of predicting systemic diseases, such as cardiovascular disease and dementia, through retinal imaging - has advanced rapidly due to the data efficiency of transformer-based foundation models like RETFound. Image-level mixed sample data augmentations, such as CutMix and MixUp, are frequently used for training transformers, yet these techniques perturb patient-specific attributes, such as medical comorbidity and clinical factors, since they only account for images and labels. To address this limitation, we propose a hierarchical sampling strategy, Oculomix, for mixed sample augmentations. Our method is based on two clinical priors. First (exam level), images acquired from the same patient at the same time point share the same attributes. Second (patient level), images acquired from the same patient at different time points have a soft temporal trend, as morbidity generally increases over time. Guided by these priors, our method constrains the mixing space to the patient and exam levels to better preserve patient-specific characteristics and leverages their hierarchical relationships. The proposed method is validated using ViT models on a five-year prediction of major adverse cardiovascular events (MACE) in a large ethnically diverse population (Alzeye). We show that Oculomix consistently outperforms image-level CutMix and MixUp by up to 3% in AUROC, demonstrating the necessity and value of the proposed method in oculomics.

Figures (3)

• Figure 1: Internal (AlzEye test set; AUROC, AUPRC) and external (HYU MACE dataset; C-index) performance of average of ViT-S and ViT-B with CutMix+MixUp under three sampling strategies: Image-level (baseline), Exam-level, and Patient+Exam-level.
• Figure 2: Overview of the proposed sampling strategy for mixed sample data augmentation (MSDA). (A) Hierarchical longitudinal ophthalmic datasets include patient-, exam-, and image-level components. (B) In hierarchical MSDA, a pairwise ranking loss captures the temporal progression between exams. $P_n$ denotes a patient, and $t_i$ and $t_j$ represent different exam time points. The prediction from mixed images at the same time point is denoted as $\hat{y}_{t_i}$, with $y_{t_i}$ as the corresponding label, while $\hat{y}_{\text{mix}}$ represents the prediction from mixed images across time points $t_i$ and $t_j$. $L_{\text{sup}}$ and $L_{\text{ranking}}$ represent supervised and ranking losses, respectively. (C) In image-level MSDA, arbitrary images are mixed regardless of patient identity or exam time point, with $y_{\text{mix}}$ and $\hat{y}_{\text{mix}}$ denoting the mixed label and its prediction.
• Figure 3: AUROC curves on the AlzEye validation dataset using CutMix+MixUp across sampling strategies.