Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Multimodal Data Processing Pipeline for MIMIC-IV Dataset

Farzana Islam Adiba, Varsha Danduri, Fahmida Liza Piya, Ali Abbasi, Mehak Gupta, Rahmatollah Beheshti

TL;DR

This work introduces a comprehensive multimodal data processing pipeline for the MIMIC-IV dataset that unifies structured EHR data, clinical notes, waveforms, chest X-ray images, and 2-D echocardiograms into a single, queryable DataFrame. It enables automated, ICD-driven cohort selection, cross-modality temporal alignment, and embedding-based representations to support arbitrary static and time-series downstream tasks, with a Python package and lightweight UI to facilitate adoption. By linking modalities through stable identifier spines and using memory-efficient raw-data pointers, the pipeline reduces preprocessing time and enhances reproducibility across studies. The approach is validated through demonstrations on mortality and ICU length-of-stay prediction across diverse cohorts, showing substantial runtime savings and broad downstream applicability, which can democratize access to robust multimodal EHR research. Overall, the pipeline expands the MIMIC ecosystem by providing a flexible, scalable, and reproducible framework for multimodal clinical data analysis.

Abstract

The MIMIC-IV dataset is a large, publicly available electronic health record (EHR) resource widely used for clinical machine learning research. It comprises multiple modalities, including structured data, clinical notes, waveforms, and imaging data. Working with these disjointed modalities requires an extensive manual effort to preprocess and align them for downstream analysis. While several pipelines for MIMIC-IV data extraction are available, they target a small subset of modalities or do not fully support arbitrary downstream applications. In this work, we greatly expand our prior popular unimodal pipeline and present a comprehensive and customizable multimodal pipeline that can significantly reduce multimodal processing time and enhance the reproducibility of MIMIC-based studies. Our pipeline systematically integrates the listed modalities, enabling automated cohort selection, temporal alignment across modalities, and standardized multimodal output formats suitable for arbitrary static and time-series downstream applications. We release the code, a simple UI, and a Python package for selective integration (with embedding) at https://github.com/healthylaife/MIMIC-IV-Data-Pipeline.

A Multimodal Data Processing Pipeline for MIMIC-IV Dataset

TL;DR

This work introduces a comprehensive multimodal data processing pipeline for the MIMIC-IV dataset that unifies structured EHR data, clinical notes, waveforms, chest X-ray images, and 2-D echocardiograms into a single, queryable DataFrame. It enables automated, ICD-driven cohort selection, cross-modality temporal alignment, and embedding-based representations to support arbitrary static and time-series downstream tasks, with a Python package and lightweight UI to facilitate adoption. By linking modalities through stable identifier spines and using memory-efficient raw-data pointers, the pipeline reduces preprocessing time and enhances reproducibility across studies. The approach is validated through demonstrations on mortality and ICU length-of-stay prediction across diverse cohorts, showing substantial runtime savings and broad downstream applicability, which can democratize access to robust multimodal EHR research. Overall, the pipeline expands the MIMIC ecosystem by providing a flexible, scalable, and reproducible framework for multimodal clinical data analysis.

Abstract

The MIMIC-IV dataset is a large, publicly available electronic health record (EHR) resource widely used for clinical machine learning research. It comprises multiple modalities, including structured data, clinical notes, waveforms, and imaging data. Working with these disjointed modalities requires an extensive manual effort to preprocess and align them for downstream analysis. While several pipelines for MIMIC-IV data extraction are available, they target a small subset of modalities or do not fully support arbitrary downstream applications. In this work, we greatly expand our prior popular unimodal pipeline and present a comprehensive and customizable multimodal pipeline that can significantly reduce multimodal processing time and enhance the reproducibility of MIMIC-based studies. Our pipeline systematically integrates the listed modalities, enabling automated cohort selection, temporal alignment across modalities, and standardized multimodal output formats suitable for arbitrary static and time-series downstream applications. We release the code, a simple UI, and a Python package for selective integration (with embedding) at https://github.com/healthylaife/MIMIC-IV-Data-Pipeline.
Paper Structure (26 sections, 4 figures, 6 tables)

This paper contains 26 sections, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Unimodal Pipelines.
  4. Multi-modal Pipelines.
  5. Method
  6. Datasets
  7. Pipeline elements
  8. Data Filtering and Cohort Selection
  9. Structured Data Processing
  10. Clinical notes processing
  11. Waveforms processing
  12. Imaging Data Processing
  13. Modalities Integration and Alignment
  14. Temporal Alignment
  15. Raw Data Integration
  16. ...and 11 more sections

Figures (4)

  • Figure 1: The overview of the proposed MIMIC-IV multimodal pipeline. The right part shows a zoomed-in version of the yellow box on the left.
  • Figure 2: The overall process of temporal alignment for different modalities.
  • Figure 3: A sample demonstration of how the database tables are integrated based on the unique identifiers.
  • Figure 4: An overview of the text extraction process from MIMIC-IV clinical notes.