Uncertainty-Calibrated Prediction of Randomly-Timed Biomarker Trajectories with Conformal Bands

TL;DR

This work tackles uncertainty in biomarker trajectory predictions when visits occur at irregular times by introducing a conformal prediction framework for randomly-timed trajectories that yields uncertainty bands with coverage $1-\alpha$. The method is model-agnostic and relies on a normalized nonconformity score to produce per-time-band intervals, providing finite-sample guarantees. It is validated on hippocampal and ventricular volume trajectories from ADNI/BLSA using multiple predictors, and extended with group-conditional bands across five covariates, plus an uncertainty-calibrated rate-of-change bound RoCB that improves high-risk subject identification. The approach offers distribution-free uncertainty quantification suitable for clinical deployment and trial design, with code available at github.com/vatass/ConformalBiomarkerTrajectories.

Abstract

Despite recent progress in predicting biomarker trajectories from real clinical data, uncertainty in the predictions poses high-stakes risks (e.g., misdiagnosis) that limit their clinical deployment. To enable safe and reliable use of such predictions in healthcare, we introduce a conformal method for uncertainty-calibrated prediction of biomarker trajectories resulting from randomly-timed clinical visits of patients. Our approach extends conformal prediction to the setting of randomly-timed trajectories via a novel nonconformity score that produces prediction bands guaranteed to cover the unknown biomarker trajectories with a user-prescribed probability. We apply our method across a wide range of standard and state-of-the-art predictors for two well-established brain biomarkers of Alzheimer's disease, using neuroimaging data from real clinical studies. We observe that our conformal prediction bands consistently achieve the desired coverage, while also being tighter than baseline prediction bands. To further account for population heterogeneity, we develop group-conditional conformal bands and test their coverage guarantees across various demographic and clinically relevant subpopulations. Moreover, we demonstrate the clinical utility of our conformal bands in identifying subjects at high risk of progression to Alzheimer's disease. Specifically, we introduce an uncertainty-calibrated risk score that enables the identification of 17.5% more high-risk subjects compared to standard risk scores, highlighting the value of uncertainty calibration in real-world clinical decision making. Our code is available at github.com/vatass/ConformalBiomarkerTrajectories.

Figures (13)

• Figure 1: Illustration of randomly-timed biomarker trajectories from five subjects. Time is measured with respect to the first biomarker observation of each subject. Note the varying trajectory length and the different time points of observations across subjects.
• Figure 2: We predict randomly-timed biomarker trajectories using arbitrary predictors (e.g., neural networks) and bound the prediction uncertainty within high-confidence prediction bands (orange band). Using these bands, we develop an uncertainty-calibrated method of identifying high-risk patients.
• Figure 3: a) We compare the mean coverage and mean interval width of our baseline and conformal prediction bands for hippocampal- and ventricular-volume trajectories. Error bars denote the 95th percentile of the metrics across 10 data splits. The baseline predictors (solid bars) either exceed the desired coverage by a noticeable margin (DKPG, DME) or fall short of it (DQR, DRMC, Bootstrap). In contrast, all conformalized predictors (striped bars) achieve the nominal coverage while also maintaining relatively tight intervals. b) We show the temporal evolution of the mean interval width of the prediction bands resulting from the conformalized DKGP predictor with $\alpha=0.1$ for hippocampal- and ventricular-volume trajectories. Notice that the bands become wider on average over time, reflecting the expected growth in uncertainty as the prediction horizon extends. Reported interval widths are on the standardized scale.
• Figure 4: We compare baseline and conformal prediction bands obtained from the DKGP and DRMC predictors for a test subject's hippocampal-volume trajectories. Our conformalized DKGP predictor produces tighter bands than its baseline counterpart. Our conformalized DRMC predictor adjusts the respective baseline prediction band in order to achieve covering the true biomarker values at all time points.
• Figure 5: We compare the mean coverage of population and group-conditional conformal prediction bands for hippocampal- and ventricular-volume trajectories. Error bars denote the 95th percentile of the metrics across 10 data splits. Results are presented across five population stratifications based on individual covariates. For most covariate groups, the population conformalized predictors fail to achieve the desired coverage, while the group-conditional conformalized predictors consistently attain the desired confidence level.

Theorems & Definitions (4)

• Theorem 3.2: Conformal Prediction Bands for Randomly-Timed Trajectories
• Remark 3.3
• Corollary 5.1: Group-Conditional Conformal Prediction Bands for Randomly-Timed Trajectories
• Definition A.1