Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sepsyn-OLCP: An Online Learning-based Framework for Early Sepsis Prediction with Uncertainty Quantification using Conformal Prediction

Anni Zhou, Beyah Raheem, Rishikesan Kamaleswaran, Yao Xie

TL;DR

Sepsyn-OLCP addresses the need for early, reliable sepsis prediction with quantified uncertainty by uniting online learning via a BayesGap-inspired contextual bandit (Online Selector) with conformal-prediction-based uncertainty (Prediction Interval Calculator). The framework uses ensemble conformal prediction (EnsembleCP0) and a gap-based Bayesian exploration strategy to adapt decision-making across multiple AI clinicians without retraining, leveraging historical EHR data and leaving-one-out calibration for uncertainty bounds. The approach demonstrates improved discrimination and balanced uncertainty quantification on the PhysioNet 2019 Sepsis Challenge dataset, with AUROC and AUPRC gains as ensemble size and exploration settings vary, and shows a practical trade-off between exploration and exploitation controlled by the significance level $\alpha$. Overall, Sepsyn-OLCP provides a scalable, uncertainty-aware online framework for early sepsis prediction with potential for real-world clinical impact through calibrated decision support and adaptive model selection.

Abstract

Sepsis is a life-threatening syndrome with high morbidity and mortality in hospitals. Early prediction of sepsis plays a crucial role in facilitating early interventions for septic patients. However, early sepsis prediction systems with uncertainty quantification and adaptive learning are scarce. This paper proposes Sepsyn-OLCP, a novel online learning algorithm for early sepsis prediction by integrating conformal prediction for uncertainty quantification and Bayesian bandits for adaptive decision-making. By combining the robustness of Bayesian models with the statistical uncertainty guarantees of conformal prediction methodologies, this algorithm delivers accurate and trustworthy predictions, addressing the critical need for reliable and adaptive systems in high-stakes healthcare applications such as early sepsis prediction. We evaluate the performance of Sepsyn-OLCP in terms of regret in stochastic bandit setting, the area under the receiver operating characteristic curve (AUROC), and F-measure. Our results show that Sepsyn-OLCP outperforms existing individual models, increasing AUROC of a neural network from 0.64 to 0.73 without retraining and high computational costs. And the model selection policy converges to the optimal strategy in the long run. We propose a novel reinforcement learning-based framework integrated with conformal prediction techniques to provide uncertainty quantification for early sepsis prediction. The proposed methodology delivers accurate and trustworthy predictions, addressing a critical need in high-stakes healthcare applications like early sepsis prediction.

Sepsyn-OLCP: An Online Learning-based Framework for Early Sepsis Prediction with Uncertainty Quantification using Conformal Prediction

TL;DR

Sepsyn-OLCP addresses the need for early, reliable sepsis prediction with quantified uncertainty by uniting online learning via a BayesGap-inspired contextual bandit (Online Selector) with conformal-prediction-based uncertainty (Prediction Interval Calculator). The framework uses ensemble conformal prediction (EnsembleCP0) and a gap-based Bayesian exploration strategy to adapt decision-making across multiple AI clinicians without retraining, leveraging historical EHR data and leaving-one-out calibration for uncertainty bounds. The approach demonstrates improved discrimination and balanced uncertainty quantification on the PhysioNet 2019 Sepsis Challenge dataset, with AUROC and AUPRC gains as ensemble size and exploration settings vary, and shows a practical trade-off between exploration and exploitation controlled by the significance level . Overall, Sepsyn-OLCP provides a scalable, uncertainty-aware online framework for early sepsis prediction with potential for real-world clinical impact through calibrated decision support and adaptive model selection.

Abstract

Sepsis is a life-threatening syndrome with high morbidity and mortality in hospitals. Early prediction of sepsis plays a crucial role in facilitating early interventions for septic patients. However, early sepsis prediction systems with uncertainty quantification and adaptive learning are scarce. This paper proposes Sepsyn-OLCP, a novel online learning algorithm for early sepsis prediction by integrating conformal prediction for uncertainty quantification and Bayesian bandits for adaptive decision-making. By combining the robustness of Bayesian models with the statistical uncertainty guarantees of conformal prediction methodologies, this algorithm delivers accurate and trustworthy predictions, addressing the critical need for reliable and adaptive systems in high-stakes healthcare applications such as early sepsis prediction. We evaluate the performance of Sepsyn-OLCP in terms of regret in stochastic bandit setting, the area under the receiver operating characteristic curve (AUROC), and F-measure. Our results show that Sepsyn-OLCP outperforms existing individual models, increasing AUROC of a neural network from 0.64 to 0.73 without retraining and high computational costs. And the model selection policy converges to the optimal strategy in the long run. We propose a novel reinforcement learning-based framework integrated with conformal prediction techniques to provide uncertainty quantification for early sepsis prediction. The proposed methodology delivers accurate and trustworthy predictions, addressing a critical need in high-stakes healthcare applications like early sepsis prediction.

Paper Structure

This paper contains 18 sections, 5 theorems, 40 equations, 11 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Materials and methods
  3. Online Selector
  4. Prediction Interval Calculator
  5. Intuition for Construction of Prediction Intervals
  6. Theoretical Analysis of the Total Probability
  7. Approximation of Prediction Intervals
  8. The Final Ensemble Prediction Interval
  9. System Workflows of Sepsyn-OLCP
  10. Theoretical Analysis of Sepsyn-OLCP
  11. Dataset Description and Preprocessing
  12. Results
  13. Experimental setup
  14. Effects of significance level $\alpha$ on Confidence Intervals and Online Exploration
  15. Complementary Strengths of Random Forest and XGBoost Combination
  16. ...and 3 more sections

Key Result

Lemma 1

Assume the event $\mathcal{E}_t$ occurs and $a_t\neq a_t^*$, which is the best arm at round $t$. Then where $B_{a_t}:=\max_{k\neq a_t} U^{\alpha}_k-L^{\alpha}_{a_t}$

Figures (11)

  • Figure 1: System Overview of Sepsyn-OLCP.
  • Figure 2: Workflows of Sepsyn-OLCP.
  • Figure 3: Average Regret (baseline = rf).
  • Figure 4: Average Regret (baseline = rf).
  • Figure 5: Average Regret (alpha = 0.25).
  • ...and 6 more figures

Theorems & Definitions (7)

  • Definition 1
  • Definition 2
  • Lemma 1: Regret Bound
  • Lemma 2: Bound on contextual gap
  • Lemma 3
  • Lemma 4: Bound of confidence interval width
  • Theorem 1: Bound on simple regret