Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Prediction of Need for Mechanical Ventilation using Cross-Attention

Anwesh Mohanty, Supreeth P. Shashikumar, Jonathan Y. Lam, Shamim Nemati

TL;DR

The study addresses predicting the need for mechanical ventilation in ICU patients by introducing FFNN-MHA, a feed-forward network enhanced with a cross-attention mechanism that uses Time-Since-Last-Measured features and patient comorbidities as queries. Trained and evaluated on the MIMIC-IV dataset, the model demonstrates a substantial performance gain, achieving a test $AUC$ of $0.9013$ and a 17.8% reduction in false positives compared to a baseline FFNN, with cross-attention outperforming self-attention. The results underscore the value of personalized contextual information and temporal feature dynamics in critical care prediction, while also highlighting the need for external validation and careful labeling. The work suggests that incorporating comorbidities into the cross-attention mechanism meaningfully enhances MV-prediction accuracy and reduces unnecessary ventilator initiation.

Abstract

In the intensive care unit, the capability to predict the need for mechanical ventilation (MV) facilitates more timely interventions to improve patient outcomes. Recent works have demonstrated good performance in this task utilizing machine learning models. This paper explores the novel application of a deep learning model with multi-head attention (FFNN-MHA) to make more accurate MV predictions and reduce false positives by learning personalized contextual information of individual patients. Utilizing the publicly available MIMIC-IV dataset, FFNN-MHA demonstrates an improvement of 0.0379 in AUC and a 17.8\% decrease in false positives compared to baseline models such as feed-forward neural networks. Our results highlight the potential of the FFNN-MHA model as an effective tool for accurate prediction of the need for mechanical ventilation in critical care settings.

Improving Prediction of Need for Mechanical Ventilation using Cross-Attention

TL;DR

The study addresses predicting the need for mechanical ventilation in ICU patients by introducing FFNN-MHA, a feed-forward network enhanced with a cross-attention mechanism that uses Time-Since-Last-Measured features and patient comorbidities as queries. Trained and evaluated on the MIMIC-IV dataset, the model demonstrates a substantial performance gain, achieving a test of and a 17.8% reduction in false positives compared to a baseline FFNN, with cross-attention outperforming self-attention. The results underscore the value of personalized contextual information and temporal feature dynamics in critical care prediction, while also highlighting the need for external validation and careful labeling. The work suggests that incorporating comorbidities into the cross-attention mechanism meaningfully enhances MV-prediction accuracy and reduces unnecessary ventilator initiation.

Abstract

In the intensive care unit, the capability to predict the need for mechanical ventilation (MV) facilitates more timely interventions to improve patient outcomes. Recent works have demonstrated good performance in this task utilizing machine learning models. This paper explores the novel application of a deep learning model with multi-head attention (FFNN-MHA) to make more accurate MV predictions and reduce false positives by learning personalized contextual information of individual patients. Utilizing the publicly available MIMIC-IV dataset, FFNN-MHA demonstrates an improvement of 0.0379 in AUC and a 17.8\% decrease in false positives compared to baseline models such as feed-forward neural networks. Our results highlight the potential of the FFNN-MHA model as an effective tool for accurate prediction of the need for mechanical ventilation in critical care settings.
Paper Structure (11 sections, 3 figures, 2 tables)

This paper contains 11 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Dataset
  4. FFNN-MHA Model
  5. Model development and training
  6. Evaluation metrics
  7. Results and Discussions
  8. Patient characteristics
  9. Performance Evaluation
  10. Interpretability analysis
  11. Conclusion

Figures (3)

  • Figure 1: Schematic diagrams for the baseline FFNN and FFNN-MHA models. (a) Baseline feed-forward neural network with the weighted TSLM layer incorporated from the COMPOSER model. (b) Proposed FFNN-MHA architecture with cross-attention implemented across the clinical features, TSLM features, and comorbidities. The cross-attention module in (b) includes a cross-attention layer followed by a layer normalization applied to the attention output. In both figures, the final output is a value between 0 and 2 indicating the risk score in a patient.
  • Figure 2: AUC plots for FFNN variations considered in this study.
  • Figure 3: Heatmap showing population level plot of contributing factors to the increase in model risk score. For example, $O_2Sat$ was identified as top contributing factor in $\sim$50% of ventilated patients 12 hours prior to $T_0$ while it was a top contributing factor in $\sim$60% of ventilated patients 1 hour prior to $T_0$. The x-axis represents hours before the onset time of MV. The y-axis represents the top factors (sorted by the magnitude of relevance score) across the patient populations.