Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Multimodal Representation Learning in Healthcare

Xiaoguang Zhu, Linxiao Gong, Lianlong Sun, Yang Liu, Haoyu Wang, Jing Liu

TL;DR

Medical multimodal learning in healthcare is hindered by systematic biases that impair generalization. The authors propose a Dual-Stream Feature Decorrelation (DFD) framework that integrates causal reasoning with bias-aware learning using a dual-stream GNN to separately model causal features $C$ and biased features $B$, and enforces decorrelation via mutual information minimization and a generalized cross-entropy loss. The approach is model-agnostic and validated on MIMIC-IV, eICU, and ADNI, where it achieves consistent performance gains over strong baselines and demonstrates effective disentanglement of causal from biased representations. By explicitly blocking spurious correlations and focusing on stable causal features, the work advances robust, equitable multimodal representations for clinical outcome prediction and sets the stage for integrating causal reasoning into broader medical AI systems.

Abstract

Medical multimodal representation learning aims to integrate heterogeneous data into unified patient representations to support clinical outcome prediction. However, real-world medical datasets commonly contain systematic biases from multiple sources, which poses significant challenges for medical multimodal representation learning. Existing approaches typically focus on effective multimodal fusion, neglecting inherent biased features that affect the generalization ability. To address these challenges, we propose a Dual-Stream Feature Decorrelation Framework that identifies and handles the biases through structural causal analysis introduced by latent confounders. Our method employs a causal-biased decorrelation framework with dual-stream neural networks to disentangle causal features from spurious correlations, utilizing generalized cross-entropy loss and mutual information minimization for effective decorrelation. The framework is model-agnostic and can be integrated into existing medical multimodal learning methods. Comprehensive experiments on MIMIC-IV, eICU, and ADNI datasets demonstrate consistent performance improvements.

Robust Multimodal Representation Learning in Healthcare

TL;DR

Medical multimodal learning in healthcare is hindered by systematic biases that impair generalization. The authors propose a Dual-Stream Feature Decorrelation (DFD) framework that integrates causal reasoning with bias-aware learning using a dual-stream GNN to separately model causal features and biased features , and enforces decorrelation via mutual information minimization and a generalized cross-entropy loss. The approach is model-agnostic and validated on MIMIC-IV, eICU, and ADNI, where it achieves consistent performance gains over strong baselines and demonstrates effective disentanglement of causal from biased representations. By explicitly blocking spurious correlations and focusing on stable causal features, the work advances robust, equitable multimodal representations for clinical outcome prediction and sets the stage for integrating causal reasoning into broader medical AI systems.

Abstract

Medical multimodal representation learning aims to integrate heterogeneous data into unified patient representations to support clinical outcome prediction. However, real-world medical datasets commonly contain systematic biases from multiple sources, which poses significant challenges for medical multimodal representation learning. Existing approaches typically focus on effective multimodal fusion, neglecting inherent biased features that affect the generalization ability. To address these challenges, we propose a Dual-Stream Feature Decorrelation Framework that identifies and handles the biases through structural causal analysis introduced by latent confounders. Our method employs a causal-biased decorrelation framework with dual-stream neural networks to disentangle causal features from spurious correlations, utilizing generalized cross-entropy loss and mutual information minimization for effective decorrelation. The framework is model-agnostic and can be integrated into existing medical multimodal learning methods. Comprehensive experiments on MIMIC-IV, eICU, and ADNI datasets demonstrate consistent performance improvements.
Paper Structure (15 sections, 9 equations, 2 figures, 2 tables)

This paper contains 15 sections, 9 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Causal Analysis
  4. Dual-Stream Graph Neural Network
  5. Causal and Biased Features Extraction
  6. Causal and Biased Features Learning
  7. Feature Decorrelation Learning
  8. Total Loss Function
  9. Experiments and results
  10. Setup
  11. Main Results
  12. Results on the MIMIC-IV and eICU Datasets
  13. Result on the ADNI Dataset
  14. Ablation Study
  15. Conclusion

Figures (2)

  • Figure 1: Overview of the proposed DFD framework. (a) The structure of the proposed dual-stream feature decorrelation network. (b) SCM in data generation view. Gray color indicates that the features are not observable. (c) SCM in the proposed method.
  • Figure 2: t-SNE visualization on MIMIC-IV readmission task.