Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Challenges and proposed solutions in modeling multimodal data: A systematic review

Maryam Farhadizadeh, Maria Weymann, Michael Blaß, Johann Kraus, Christopher Gundler, Sebastian Walter, Noah Hempen, Harald Binder, Nadine Binder

TL;DR

This systematic review synthesizes findings from 69 studies to identify common obstacles, including missing modalities, limited sample sizes, dimensionality imbalance, interpretability issues, and finding the optimal fusion techniques.

Abstract

Multimodal data modeling has emerged as a powerful approach in clinical research, enabling the integration of diverse data types such as imaging, genomics, wearable sensors, and electronic health records. Despite its potential to improve diagnostic accuracy and support personalized care, modeling such heterogeneous data presents significant technical challenges. This systematic review synthesizes findings from 69 studies to identify common obstacles, including missing modalities, limited sample sizes, dimensionality imbalance, interpretability issues, and finding the optimal fusion techniques. We highlight recent methodological advances, such as transfer learning, generative models, attention mechanisms, and neural architecture search that offer promising solutions. By mapping current trends and innovations, this review provides a comprehensive overview of the field and offers practical insights to guide future research and development in multimodal modeling for medical applications.

Challenges and proposed solutions in modeling multimodal data: A systematic review

TL;DR

This systematic review synthesizes findings from 69 studies to identify common obstacles, including missing modalities, limited sample sizes, dimensionality imbalance, interpretability issues, and finding the optimal fusion techniques.

Abstract

Multimodal data modeling has emerged as a powerful approach in clinical research, enabling the integration of diverse data types such as imaging, genomics, wearable sensors, and electronic health records. Despite its potential to improve diagnostic accuracy and support personalized care, modeling such heterogeneous data presents significant technical challenges. This systematic review synthesizes findings from 69 studies to identify common obstacles, including missing modalities, limited sample sizes, dimensionality imbalance, interpretability issues, and finding the optimal fusion techniques. We highlight recent methodological advances, such as transfer learning, generative models, attention mechanisms, and neural architecture search that offer promising solutions. By mapping current trends and innovations, this review provides a comprehensive overview of the field and offers practical insights to guide future research and development in multimodal modeling for medical applications.
Paper Structure (2 sections, 4 figures, 5 tables)

This paper contains 2 sections, 4 figures, 5 tables.

Table of Contents

  1. .
  2. .

Figures (4)

  • Figure 1: Multimodal Modeling: Various information sources can be collected from a patient during a clinical stay or a doctor's visit resulting in different data modalities. Subsequently, the modalities can be evaluated jointly using a suitable fusion technique, where the fusion (F) can be happening at different stages in the process.
  • Figure 2: PRISMA flow diagram illustrating the study selection process following the systematic search in PubMed. In addition, 11 relevant studies were identified through a manual Google Scholar search, resulting in a total of 69 included studies.
  • Figure 3: Histogram of key challenges addressed in yearly publications. Studies may address multiple challenges.
  • Figure 4: Distribution of analysis tasks across data modality combinations. Bars show the number of studies per data modality, subdivided by task type.