Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey on Trustworthiness in Foundation Models for Medical Image Analysis

Congzhen Shi, Ryan Rezai, Jiaxi Yang, Qi Dou, Xiaoxiao Li

TL;DR

This survey aims to fill the gap for advancing towards trustworthy AI in medical image analysis by presenting a novel taxonomy of foundation models used in medical imaging and analyzing the key motivations for ensuring their trustworthiness, and reviewing current research on foundation models in major medical imaging applications.

Abstract

The rapid advancement of foundation models in medical imaging represents a significant leap toward enhancing diagnostic accuracy and personalized treatment. However, the deployment of foundation models in healthcare necessitates a rigorous examination of their trustworthiness, encompassing privacy, robustness, reliability, explainability, and fairness. The current body of survey literature on foundation models in medical imaging reveals considerable gaps, particularly in the area of trustworthiness. Additionally, existing surveys on the trustworthiness of foundation models do not adequately address their specific variations and applications within the medical imaging domain. This survey aims to fill that gap by presenting a novel taxonomy of foundation models used in medical imaging and analyzing the key motivations for ensuring their trustworthiness. We review current research on foundation models in major medical imaging applications, focusing on segmentation, medical report generation, medical question and answering (Q\&A), and disease diagnosis. These areas are highlighted because they have seen a relatively mature and substantial number of foundation models compared to other applications. We focus on literature that discusses trustworthiness in medical image analysis manuscripts. We explore the complex challenges of building trustworthy foundation models for each application, summarizing current concerns and strategies for enhancing trustworthiness. Furthermore, we examine the potential of these models to revolutionize patient care. Our analysis underscores the imperative for advancing towards trustworthy AI in medical image analysis, advocating for a balanced approach that fosters innovation while ensuring ethical and equitable healthcare delivery.

A Survey on Trustworthiness in Foundation Models for Medical Image Analysis

TL;DR

This survey aims to fill the gap for advancing towards trustworthy AI in medical image analysis by presenting a novel taxonomy of foundation models used in medical imaging and analyzing the key motivations for ensuring their trustworthiness, and reviewing current research on foundation models in major medical imaging applications.

Abstract

The rapid advancement of foundation models in medical imaging represents a significant leap toward enhancing diagnostic accuracy and personalized treatment. However, the deployment of foundation models in healthcare necessitates a rigorous examination of their trustworthiness, encompassing privacy, robustness, reliability, explainability, and fairness. The current body of survey literature on foundation models in medical imaging reveals considerable gaps, particularly in the area of trustworthiness. Additionally, existing surveys on the trustworthiness of foundation models do not adequately address their specific variations and applications within the medical imaging domain. This survey aims to fill that gap by presenting a novel taxonomy of foundation models used in medical imaging and analyzing the key motivations for ensuring their trustworthiness. We review current research on foundation models in major medical imaging applications, focusing on segmentation, medical report generation, medical question and answering (Q\&A), and disease diagnosis. These areas are highlighted because they have seen a relatively mature and substantial number of foundation models compared to other applications. We focus on literature that discusses trustworthiness in medical image analysis manuscripts. We explore the complex challenges of building trustworthy foundation models for each application, summarizing current concerns and strategies for enhancing trustworthiness. Furthermore, we examine the potential of these models to revolutionize patient care. Our analysis underscores the imperative for advancing towards trustworthy AI in medical image analysis, advocating for a balanced approach that fosters innovation while ensuring ethical and equitable healthcare delivery.
Paper Structure (42 sections, 4 figures, 2 tables)

This paper contains 42 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Adaptations of Foundation Models in Medical Imaging
  3. Definition of Foundation Models
  4. Different Methods for Adapting Foundation Models in Medical Imaging
  5. Training From Scratch
  6. Fine-Tuning
  7. Prompt-Tuning
  8. Off-the-Shelf
  9. Trustworthiness Perspectives in Medical Image Analysis
  10. Privacy (T1)
  11. Robustness (T2)
  12. Reliability (T3)
  13. Explainability (T4)
  14. Fairness (T5)
  15. Unpacking Trustworthiness in Current Foundation Models Techniques for Medical Imaging Tasks
  16. ...and 27 more sections

Figures (4)

  • Figure 1: A schematic overview of motivations, foundation model usage, tasks and trustworthiness enhancements discussed in this paper.
  • Figure 2: Roadmap of this paper.
  • Figure 3: Landscape of foundation models, their usage, and trustworthiness issues for medical image analysis. The relationship between FM usage and trustworthiness issues for different applications is shown. Each thread represents one piece of literature reviewed in this survey.
  • Figure 4: Word cloud visualization of questions (left) and answers (right) from the training set of the PMC-VQA dataset.