Explainable Artificial Intelligence Techniques for Irregular Temporal Classification of Multidrug Resistance Acquisition in Intensive Care Unit Patients

TL;DR

This work tackles MDR acquisition prediction in ICU patients using irregular MTS derived from EHR data. It introduces a GRU-based temporal classifier that handles irregular sampling via masking and is augmented by intrinsic Hadamard attention and post-hoc IT-SHAP explanations, enabling time-varying interpretability of predictions. The study demonstrates strong predictive performance (ROC AUC up to $78.27\%$) and clinically relevant insights into risk factors such as prior non-resistant cultures and antibiotic usage, while highlighting advantages and limitations of different XAI approaches. Practically, the framework supports early interventions (e.g., preventive isolation) and informs tailored treatments, with potential applicability to other clinical temporal classification tasks modeled as irregular MTS.

Abstract

Antimicrobial Resistance represents a significant challenge in the Intensive Care Unit (ICU), where patients are at heightened risk of Multidrug-Resistant (MDR) infections-pathogens resistant to multiple antimicrobial agents. This study introduces a novel methodology that integrates Gated Recurrent Units (GRUs) with advanced intrinsic and post-hoc interpretability techniques for detecting the onset of MDR in patients across time. Within interpretability methods, we propose Explainable Artificial Intelligence (XAI) approaches to handle irregular Multivariate Time Series (MTS), introducing Irregular Time Shapley Additive Explanations (IT-SHAP), a modification of Shapley Additive Explanations designed for irregular MTS with Recurrent Neural Networks focused on temporal outputs. Our methodology aims to identify specific risk factors associated with MDR in ICU patients. GRU with Hadamard's attention demonstrated high initial specificity and increasing sensitivity over time, correlating with increased nosocomial infection risks during prolonged ICU stays. XAI analysis, enhanced by Hadamard attention and IT-SHAP, identified critical factors such as previous non-resistant cultures, specific antibiotic usage patterns, and hospital environment dynamics. These insights suggest that early detection of at-risk patients can inform interventions such as preventive isolation and customized treatments, significantly improving clinical outcomes. The proposed GRU model for temporal classification achieved an average Receiver Operating Characteristic Area Under the Curve of 78.27 +- 1.26 over time, indicating strong predictive performance. In summary, this study highlights the clinical utility of our methodology, which combines predictive accuracy with interpretability, thereby facilitating more effective healthcare interventions by professionals.

Figures (7)

• Figure 1: Graphical illustration of the proposed workflow. The process begins with data from Electronic Health Records (EHR), which is modeled into an irregular Multivariate Time Series (MTS). The Temporal Balanced Binary Cross Entropy (TBBCE) is applied to manage class imbalance and temporal dependencies. The pre-processed data is then input into the Gate Recurrent Unit (GRU) model. For interpretability, XAI techniques including Feature Selection (FS), Hadamard, and Irregular Time Shapley Additive Explanations (IT-SHAP) are utilized for temporal outputs. Finally, model performance is evaluated using metrics that include the Receiver Operating Characteristic Area Under the Curve (ROC AUC), Specificity, and Sensitivity.
• Figure 2: Preprocessing steps for MTS data. The first figure (top) represents the data for an MDR patient, where $\mathbf{y}_i$ indicates the time-varying MDR label, and $\mathbf{X}_{i}$ collects the values of the $F$ features for the $T=14$ time instants. The second figure (bottom) represent the data for a non-MDR patient. The timeline spans from ICU admission (8 AM) to ICU discharge, with a 24-hour sampling period. For MDR patients, if the culture is flagged positive on the day $t_i$, then $[\mathbf{y}_i]_{(t)} = 0$ for $t<t_i$ and $[\mathbf{y}_i]_{(t)} = 1$ for $t\geq t_i$. For non-MDR patients $[\mathbf{y}_i]_{(t)} = 0$ throughout their ICU stay.
• Figure 3: Performance of the predictive model over time. (a) GRU model performance, and (b) GRU model with an attention mechanism. Each plot displays the mean and standard deviation of specificity, sensitivity, and ROC AUC across different time points.
• Figure 4: Pre-hoc interpretability using CMI. (a) Importance scores for the entire population; (b) Importance scores for MDR patients; and (c) Importance scores for non-MDR patients. The x-axis represents the time steps, and the y-axis represents the variables. The color scale indicates the importance of each variable at each time step.
• Figure 5: Intrinsic interpretability using Hadamard attention mechanism. (a) Attention weights for the entire population; (b) Attention weights for MDR patients; and (c) Attention weights for non-MDR patients. The x-axis represents the time steps, and the y-axis represents the variables. The color scale indicates the importance of each variable at each time step.