Learning to Diagnose with LSTM Recurrent Neural Networks

TL;DR

This work demonstrates that LSTM RNNs can learn meaningful patterns from irregular multivariate ICU time series to perform multilabel diagnosis. It introduces sequential target replication and auxiliary outputs as regularization strategies, showing that these techniques improve predictive performance over strong baselines, including an MLP with hand-engineered features. The results indicate that LSTMs can outperform traditional approaches on a comprehensive PICU dataset, with ensembles offering additional gains. The study also discusses limitations related to data preprocessing and proposes future directions for early diagnosis and improved handling of missing data and interpretability.

Abstract

Clinical medical data, especially in the intensive care unit (ICU), consist of multivariate time series of observations. For each patient visit (or episode), sensor data and lab test results are recorded in the patient's Electronic Health Record (EHR). While potentially containing a wealth of insights, the data is difficult to mine effectively, owing to varying length, irregular sampling and missing data. Recurrent Neural Networks (RNNs), particularly those using Long Short-Term Memory (LSTM) hidden units, are powerful and increasingly popular models for learning from sequence data. They effectively model varying length sequences and capture long range dependencies. We present the first study to empirically evaluate the ability of LSTMs to recognize patterns in multivariate time series of clinical measurements. Specifically, we consider multilabel classification of diagnoses, training a model to classify 128 diagnoses given 13 frequently but irregularly sampled clinical measurements. First, we establish the effectiveness of a simple LSTM network for modeling clinical data. Then we demonstrate a straightforward and effective training strategy in which we replicate targets at each sequence step. Trained only on raw time series, our models outperform several strong baselines, including a multilayer perceptron trained on hand-engineered features.

Figures (6)

• Figure 1: A simple RNN model for multilabel classification. Green rectangles represent inputs. The recurrent hidden layers separating input and output are represented with a single blue rectangle. The red rectangle represents targets.
• Figure 2: An RNN classification model with target replication. The primary target (depicted in red) at the final step is used at prediction time, but during training, the model back-propagates errors from the intermediate targets (purple) at every sequence step.
• Figure 3: Our dataset contains many labels. For our task, a subset of $128$ are of interest (depicted in red). Our Auiliary Output neural network makes use of extra labels as additional training targets (depicted in purple). At inference time we generate predictions for only the labels of interest.
• Figure 4: Training curves showing the impact of the DO, AO, and TR strategies on overfitting.
• Figure 5: Each chart depicts the probabilities assigned by each of four models at each (hourly re-sampled) time step. LSTM-Simple uses only targets at the final time step. LSTM-TR uses target replication. LSTM-AO uses auxiliary outputs (diagnoses), and LSTM-TR,AO uses both techniques. LSTMs with target replication learn to make accurate diagnoses earlier.