LEARNER: Contrastive Pretraining for Learning Fine-Grained Patient Progression from Coarse Inter-Patient Labels

TL;DR

LEARNER addresses the challenge of predicting subtle intra-patient progression from limited longitudinal data by leveraging coarse inter-patient differences through contrastive pretraining. The method combines single-scan $HS$ regression with an inter-patient contrastive objective and uses a Temporal Shift Module backbone to learn patient-specific embeddings, employing the embedding difference $d_t = u_{t+1}-u_t$ for downstream classification. The approach yields improved three-way health-change prediction on both lung ultrasound ($S/F$ ratio) and ADNI MRI ($MMSE$) data, with the weighted contrastive loss providing the largest gains and better alignment between embedding distances and clinical changes. These results suggest that inter-patient contrastive learning can enable robust, individualized outcome predictions from scarce longitudinal data, with potential applicability to broader imaging domains and larger cohorts.

Abstract

Predicting whether a treatment leads to meaningful improvement is a central challenge in personalized medicine, particularly when disease progression manifests as subtle visual changes over time. While data-driven deep learning (DL) offers a promising route to automate such predictions, acquiring large-scale longitudinal data for each individual patient remains impractical. To address this limitation, we explore whether inter-patient variability can serve as a proxy for learning intra-patient progression. We propose LEARNER, a contrastive pretraining framework that leverages coarsely labeled inter-patient data to learn fine-grained, patient-specific representations. Using lung ultrasound (LUS) and brain MRI datasets, we demonstrate that contrastive objectives trained on coarse inter-patient differences enable models to capture subtle intra-patient changes associated with treatment response. Across both modalities, our approach improves downstream classification accuracy and F1-score compared to standard MSE pretraining, highlighting the potential of inter-patient contrastive learning for individualized outcome prediction.

Figures (3)

• Figure 1: Comparison of embedding distance versus the actual change in physiological health score ($\Delta$S/F ratio) between day-1 and day-2 scans of individual patients. Each point represents a scan pair from the same patient, where the $x$-axis is the absolute difference in ground-truth S/F ratio between the two scans (larger values indicate greater clinical improvement or decline), and the $y$-axis (Embedding Distance) is the cosine distance between their latent embeddings. With MSE pretraining, many pairs with large S/F ratio differences still have small embedding distances (points clustered near the bottom-right), indicating that the representation underestimates clinical change. Adding contrastive pretraining shifts pairs with larger label differences upward and yields a more spread-out, approximately increasing pattern, with the weighted contrastive variant producing the strongest separation. This behavior suggests that contrastive pretraining aligns embedding distances more faithfully with clinical differences and improves sensitivity to subtle physiological progression.
• Figure 1: Contrastive Learning based Pre-training
• Figure 2: Proposed Model Architecture