Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Applications of Large Models in Medicine

YunHe Su, Zhengyang Lu, Junhui Liu, Ke Pang, Haoran Dai, Sa Liu, Yuxin Jia, Lujia Ge, Jing-min Yang

TL;DR

The paper surveys how Medical Large Models (MedLMs) spanning LLMs, Vision, 3D, Multimodal, and Large Graph Models are transforming medicine by enabling disease prediction, diagnosis, personalized treatment, and drug discovery. It covers dataset sources, training and evaluation strategies, and domain-specific challenges, highlighting advances in LLMs for clinical reasoning, Vision-Language models for image-text tasks, 3D modeling for anatomy and implants, and graph-based approaches for brain networks, knowledge graphs, and molecular interactions; notable examples include MedPaLM, BiomedCLIP, AnomalyCLIP, TxGNN, and GraphDTA, with performance metrics such as $AUC$-ROC reaching competitive levels in several tasks. The discussion emphasizes the importance of data quality, safety, interpretability, and human-in-the-loop design, outlining future directions toward efficient, domain-aware, andExplainable AI that integrates diverse data modalities for precision medicine. Overall, the surveyed landscape suggests substantial potential for improving diagnostic accuracy, treatment personalization, and drug discovery, while also underscoring the need for rigorous validation, ethical considerations, and seamless clinical integration to realize real-world impact.

Abstract

This paper explores the advancements and applications of large-scale models in the medical field, with a particular focus on Medical Large Models (MedLMs). These models, encompassing Large Language Models (LLMs), Vision Models, 3D Large Models, and Multimodal Models, are revolutionizing healthcare by enhancing disease prediction, diagnostic assistance, personalized treatment planning, and drug discovery. The integration of graph neural networks in medical knowledge graphs and drug discovery highlights the potential of Large Graph Models (LGMs) in understanding complex biomedical relationships. The study also emphasizes the transformative role of Vision-Language Models (VLMs) and 3D Large Models in medical image analysis, anatomical modeling, and prosthetic design. Despite the challenges, these technologies are setting new benchmarks in medical innovation, improving diagnostic accuracy, and paving the way for personalized healthcare solutions. This paper aims to provide a comprehensive overview of the current state and future directions of large models in medicine, underscoring their significance in advancing global health.

Applications of Large Models in Medicine

TL;DR

The paper surveys how Medical Large Models (MedLMs) spanning LLMs, Vision, 3D, Multimodal, and Large Graph Models are transforming medicine by enabling disease prediction, diagnosis, personalized treatment, and drug discovery. It covers dataset sources, training and evaluation strategies, and domain-specific challenges, highlighting advances in LLMs for clinical reasoning, Vision-Language models for image-text tasks, 3D modeling for anatomy and implants, and graph-based approaches for brain networks, knowledge graphs, and molecular interactions; notable examples include MedPaLM, BiomedCLIP, AnomalyCLIP, TxGNN, and GraphDTA, with performance metrics such as -ROC reaching competitive levels in several tasks. The discussion emphasizes the importance of data quality, safety, interpretability, and human-in-the-loop design, outlining future directions toward efficient, domain-aware, andExplainable AI that integrates diverse data modalities for precision medicine. Overall, the surveyed landscape suggests substantial potential for improving diagnostic accuracy, treatment personalization, and drug discovery, while also underscoring the need for rigorous validation, ethical considerations, and seamless clinical integration to realize real-world impact.

Abstract

This paper explores the advancements and applications of large-scale models in the medical field, with a particular focus on Medical Large Models (MedLMs). These models, encompassing Large Language Models (LLMs), Vision Models, 3D Large Models, and Multimodal Models, are revolutionizing healthcare by enhancing disease prediction, diagnostic assistance, personalized treatment planning, and drug discovery. The integration of graph neural networks in medical knowledge graphs and drug discovery highlights the potential of Large Graph Models (LGMs) in understanding complex biomedical relationships. The study also emphasizes the transformative role of Vision-Language Models (VLMs) and 3D Large Models in medical image analysis, anatomical modeling, and prosthetic design. Despite the challenges, these technologies are setting new benchmarks in medical innovation, improving diagnostic accuracy, and paving the way for personalized healthcare solutions. This paper aims to provide a comprehensive overview of the current state and future directions of large models in medicine, underscoring their significance in advancing global health.

Paper Structure

This paper contains 26 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. LLMs in Medical
  3. Large Vision Models in Medical
  4. Visual Anomaly Detection in Medical Images
  5. Vision-Language Models for Zero-shot Medical Anomaly Detection
  6. Diffusion Models for Unsupervised Detection
  7. Self-supervised Learning for Anomaly Detection
  8. Semi-supervised Learning Approaches
  9. Challenges and Future Directions
  10. Applications in Pathological Images.
  11. Image Segmentation with U-Net
  12. Image Generation with GANs
  13. Anomaly Detection in Pathological Images
  14. Applications of Large Multimodal Model in Medical
  15. VLM for Medical Image Analysis
  16. ...and 11 more sections

Figures (2)

  • Figure 1: The overall structure of the survey.
  • Figure 2: Evolution timeline of Large Models and their applications in medical.Including language models, vision models, multi-modal models and graph-based models.