Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bridging Electronic Health Records and Clinical Texts: Contrastive Learning for Enhanced Clinical Tasks

Sara Ketabi, Dhanesh Ramachandram

TL;DR

The paper addresses the bottleneck of limited semantic information in structured EHR data for complex clinical tasks. It proposes a deep multimodal contrastive learning framework that aligns static-form EHR features with discharge summaries using a TabNet encoder for EHR and Longformer for text, trained with a CLIP-style objective, followed by downstream fine-tuning on two high-impact prediction tasks. The approach yields notable improvements in AUROC over strong baselines, with particularly large gains under data-scarce conditions, and demonstrates that CL-based pretraining can serve as a robust initialization for tabular clinical analyses even when text is not required at inference. The framework is modular and adaptable to additional modalities, offering a practical path to richer, context-aware clinical decision support systems in real-world hospital settings.

Abstract

Conventional machine learning models, particularly tree-based approaches, have demonstrated promising performance across various clinical prediction tasks using electronic health record (EHR) data. Despite their strengths, these models struggle with tasks that require deeper contextual understanding, such as predicting 30-day hospital readmission. This can be primarily due to the limited semantic information available in structured EHR data. To address this limitation, we propose a deep multimodal contrastive learning (CL) framework that aligns the latent representations of structured EHR data with unstructured discharge summary notes. It works by pulling together paired EHR and text embeddings while pushing apart unpaired ones. Fine-tuning the pretrained EHR encoder extracted from this framework significantly boosts downstream task performance, e.g., a 4.1% AUROC enhancement over XGBoost for 30-day readmission prediction. Such results demonstrate the effect of integrating domain knowledge from clinical notes into EHR-based pipelines, enabling more accurate and context-aware clinical decision support systems.

Bridging Electronic Health Records and Clinical Texts: Contrastive Learning for Enhanced Clinical Tasks

TL;DR

The paper addresses the bottleneck of limited semantic information in structured EHR data for complex clinical tasks. It proposes a deep multimodal contrastive learning framework that aligns static-form EHR features with discharge summaries using a TabNet encoder for EHR and Longformer for text, trained with a CLIP-style objective, followed by downstream fine-tuning on two high-impact prediction tasks. The approach yields notable improvements in AUROC over strong baselines, with particularly large gains under data-scarce conditions, and demonstrates that CL-based pretraining can serve as a robust initialization for tabular clinical analyses even when text is not required at inference. The framework is modular and adaptable to additional modalities, offering a practical path to richer, context-aware clinical decision support systems in real-world hospital settings.

Abstract

Conventional machine learning models, particularly tree-based approaches, have demonstrated promising performance across various clinical prediction tasks using electronic health record (EHR) data. Despite their strengths, these models struggle with tasks that require deeper contextual understanding, such as predicting 30-day hospital readmission. This can be primarily due to the limited semantic information available in structured EHR data. To address this limitation, we propose a deep multimodal contrastive learning (CL) framework that aligns the latent representations of structured EHR data with unstructured discharge summary notes. It works by pulling together paired EHR and text embeddings while pushing apart unpaired ones. Fine-tuning the pretrained EHR encoder extracted from this framework significantly boosts downstream task performance, e.g., a 4.1% AUROC enhancement over XGBoost for 30-day readmission prediction. Such results demonstrate the effect of integrating domain knowledge from clinical notes into EHR-based pipelines, enabling more accurate and context-aware clinical decision support systems.

Paper Structure

This paper contains 16 sections, 7 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Deep Learning Architectures for Tabular Data Processing
  4. Multimodal Contrastive Learning Frameworks
  5. Methodology
  6. EHR-Text Contrastive Learning
  7. Downstream Tasks
  8. Experiments and Results
  9. Dataset
  10. Implementation Details
  11. Evaluation Setting
  12. Results
  13. Hospital Readmission Prediction
  14. Critical Outcome Prediction
  15. Discussion
  16. ...and 1 more sections

Figures (2)

  • Figure 1: The Proposed EHR-Text CL Framework: EHR is encoded with TabNet initialized with masked pretraining and then forwarded to a linear layer to get its representation. Each discharge summary is divided into chunks of 256 tokens, each encoded by a separate Longformer transformer. The mean of the CLS embeddings extracted from the last layer of all longformer models is then passed through a linear layer to obtain the text representation. A contrastive loss (CLIP-style) is applied between the EHR and text representations to align the related pairs and train the framework.
  • Figure 2: The Downstream Classification Architecture: The TabNet encoder is initialized with the proposed CL weights. The initial embedding and feature-splitting layers are kept frozen, and the remaining layers along with two fully-connected layers are fine-tuned for each downstream task.