Fast and Interpretable Mortality Risk Scores for Critical Care Patients

TL;DR

Group Faster Risk is a fast, accessible, and flexible procedure that allows learning a diverse set of sparse risk scores for mortality prediction and performs better than risk scores currently used in hospitals, and on par with black box ML models, while being orders of magnitude sparser.

Abstract

Prediction of mortality in intensive care unit (ICU) patients typically relies on black box models (that are unacceptable for use in hospitals) or hand-tuned interpretable models (that might lead to the loss in performance). We aim to bridge the gap between these two categories by building on modern interpretable ML techniques to design interpretable mortality risk scores that are as accurate as black boxes. We developed a new algorithm, GroupFasterRisk, which has several important benefits: it uses both hard and soft direct sparsity regularization, it incorporates group sparsity to allow more cohesive models, it allows for monotonicity constraint to include domain knowledge, and it produces many equally-good models, which allows domain experts to choose among them. For evaluation, we leveraged the largest existing public ICU monitoring datasets (MIMIC III and eICU). Models produced by GroupFasterRisk outperformed OASIS and SAPS II scores and performed similarly to APACHE IV/IVa while using at most a third of the parameters. For patients with sepsis/septicemia, acute myocardial infarction, heart failure, and acute kidney failure, GroupFasterRisk models outperformed OASIS and SOFA. Finally, different mortality prediction ML approaches performed better based on variables selected by GroupFasterRisk as compared to OASIS variables. GroupFasterRisk's models performed better than risk scores currently used in hospitals, and on par with black box ML models, while being orders of magnitude sparser. Because GroupFasterRisk produces a variety of risk scores, it allows design flexibility - the key enabler of practical model creation. GroupFasterRisk is a fast, accessible, and flexible procedure that allows learning a diverse set of sparse risk scores for mortality prediction.

Figures (29)

• Figure 1: Risk score produced by GroupFasterRisk. This model has a group sparsity of 15 (GFR-15), which means that the model uses 15 variables with multiple splits per variable, which create that variable's component function. The total number of splits (overall sparsity) is regularized as well as the total number of variables(group sparsity). Max and Min represent the maximum and minimum value of the measurement over the first 24 hours of a patient's ICU stay. We applied monotonicity constraint to Max Bilirubin, Max BUN, Min GCS, and Min SBP as we discuss in Supplementary Material \ref{['appendix:monoton_correction_fr15']}.
• Figure 2: GroupFasterRisk algorithm workflow. We first find a near-optimal solution for a sparse logistic regression problem without the integer constraints. (Beam search involves finding $B$ solutions at each iteration before arriving at the final solution.) This solution is used in the second stage to search for a diverse pool of sparse continuous solutions that also satisfy various constraints while having similar predictive accuracy. We subsequently select the top $M$ solutions and apply a rounding search subroutine to obtain integer-valued solutions. Our algorithm is carefully designed to ensure that the integer-valued solutions maintain similar performance to real-valued solutions.
• Figure 3: Comparison of GroupFasterRisk models with OASIS, SAPS II, APACHE IV, and APACHE IVa on all-cause in-hospital mortality prediction task.
• Figure 4: Group sparsities and time consumption of GroupFasterRisk.
• Figure 5: Evaluation on disease-specific cohorts.