Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient MedSAMs: Segment Anything in Medical Images on Laptop

Jun Ma, Feifei Li, Sumin Kim, Reza Asakereh, Bao-Hiep Le, Dang-Khoa Nguyen-Vu, Alexander Pfefferle, Muxin Wei, Ruochen Gao, Donghang Lyu, Songxiao Yang, Lennart Purucker, Zdravko Marinov, Marius Staring, Haisheng Lu, Thuy Thanh Dao, Xincheng Ye, Zhi Li, Gianluca Brugnara, Philipp Vollmuth, Martha Foltyn-Dumitru, Jaeyoung Cho, Mustafa Ahmed Mahmutoglu, Martin Bendszus, Irada Pflüger, Aditya Rastogi, Dong Ni, Xin Yang, Guang-Quan Zhou, Kaini Wang, Nicholas Heller, Nikolaos Papanikolopoulos, Christopher Weight, Yubing Tong, Jayaram K Udupa, Cahill J. Patrick, Yaqi Wang, Yifan Zhang, Francisco Contijoch, Elliot McVeigh, Xin Ye, Shucheng He, Robert Haase, Thomas Pinetz, Alexander Radbruch, Inga Krause, Erich Kobler, Jian He, Yucheng Tang, Haichun Yang, Yuankai Huo, Gongning Luo, Kaisar Kushibar, Jandos Amankulov, Dias Toleshbayev, Amangeldi Mukhamejan, Jan Egger, Antonio Pepe, Christina Gsaxner, Gijs Luijten, Shohei Fujita, Tomohiro Kikuchi, Benedikt Wiestler, Jan S. Kirschke, Ezequiel de la Rosa, Federico Bolelli, Luca Lumetti, Costantino Grana, Kunpeng Xie, Guomin Wu, Behrus Puladi, Carlos Martín-Isla, Karim Lekadir, Victor M. Campello, Wei Shao, Wayne Brisbane, Hongxu Jiang, Hao Wei, Wu Yuan, Shuangle Li, Yuyin Zhou, Bo Wang

TL;DR

The first international competition dedicated to promptable medical image segmentation, featuring a large-scale dataset spanning nine common imaging modalities from over 20 different institutions, and developed lightweight segmentation foundation models and implemented an efficient inference pipeline that substantially reduced computational requirements while maintaining state-of-the-art segmentation accuracy.

Abstract

Promptable segmentation foundation models have emerged as a transformative approach to addressing the diverse needs in medical images, but most existing models require expensive computing, posing a big barrier to their adoption in clinical practice. In this work, we organized the first international competition dedicated to promptable medical image segmentation, featuring a large-scale dataset spanning nine common imaging modalities from over 20 different institutions. The top teams developed lightweight segmentation foundation models and implemented an efficient inference pipeline that substantially reduced computational requirements while maintaining state-of-the-art segmentation accuracy. Moreover, the post-challenge phase advanced the algorithms through the design of performance booster and reproducibility tasks, resulting in improved algorithms and validated reproducibility of the winning solution. Furthermore, the best-performing algorithms have been incorporated into the open-source software with a user-friendly interface to facilitate clinical adoption. The data and code are publicly available to foster the further development of medical image segmentation foundation models and pave the way for impactful real-world applications.

Efficient MedSAMs: Segment Anything in Medical Images on Laptop

TL;DR

The first international competition dedicated to promptable medical image segmentation, featuring a large-scale dataset spanning nine common imaging modalities from over 20 different institutions, and developed lightweight segmentation foundation models and implemented an efficient inference pipeline that substantially reduced computational requirements while maintaining state-of-the-art segmentation accuracy.

Abstract

Promptable segmentation foundation models have emerged as a transformative approach to addressing the diverse needs in medical images, but most existing models require expensive computing, posing a big barrier to their adoption in clinical practice. In this work, we organized the first international competition dedicated to promptable medical image segmentation, featuring a large-scale dataset spanning nine common imaging modalities from over 20 different institutions. The top teams developed lightweight segmentation foundation models and implemented an efficient inference pipeline that substantially reduced computational requirements while maintaining state-of-the-art segmentation accuracy. Moreover, the post-challenge phase advanced the algorithms through the design of performance booster and reproducibility tasks, resulting in improved algorithms and validated reproducibility of the winning solution. Furthermore, the best-performing algorithms have been incorporated into the open-source software with a user-friendly interface to facilitate clinical adoption. The data and code are publicly available to foster the further development of medical image segmentation foundation models and pave the way for impactful real-world applications.

Paper Structure

This paper contains 18 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Results
  3. Discussion
  4. Methods

Figures (5)

  • Figure 1: Competition design.a, The task is to develop universal segmentation foundation models that can accept various medical image inputs with target bounding box prompts and generate the corresponding segmentation masks. The model should be lightweight and deployable on laptops without reliance on graphics processing unit (GPU). b, Three phases in the competition. During the development phase, participants train their models on the training set and obtain performance metrics on the online validation set on Codabench. The top 20 teams on the validation leaderboard are invited to submit their algorithm dockers and we manually evaluate them on the hidden testing set. After that, we release all the technical details and code of the top ten teams and launch a post-challenge phase to invite participants to further boost their model performance and reproduce the winning algorithms.
  • Figure 2: Competition dataset.a, Example images and the annotations in the training and testing set. b, The number of image-mask pairs in the public training set. c, The number of image-mask pairs in the hidden testing set. All testing data has been newly collected specifically for this challenge and was not publicly available beforehand. d, Geographical distribution of participants and data contributors in the testing set.
  • Figure 3: Evaluation results of 23 algorithms on the hidden testing set. Dot and box plot of the Average DSC and NSD scores on the a, 2D (n = 2,309 images) and b, 3D (n = 2,105 scans) testing set. The box plots display descriptive statistics across all testing cases, with the median value represented by the horizontal line within the box, the lower and upper quartiles delineating the borders of the box and the vertical black lines indicating 1.5 × IQR. The algorithms are organized on the x-axis based on their corresponding ranks. The bubble plots show the trade-off between segmentation accuracy (DSC and NSD) and efficiency on the c, 2D and d, 3D testing set. The circle size is proportional to the NSD score. e, Modality-wise segmentation accuracy performance (Average DSC and NSD) of three best-performing teams and LiteMedSAM baseline. Modality-wise segmentation efficiency (runtime) of three best-performing teams and LiteMedSAM baseline on 3D f, and 2D g, modalities. h, Visualized segmentation results of nine modalities. The blue bounding box and green overlay denote prompt and reference standard, respectively. For each image, the best DSC score from the three algorithms and the corresponding contour are presented.
  • Figure 4: Results of the post-challenge.a, The new best-performing algorithm (T2-automlfreiburg-post) and comparison to its predecessor (T2-automlfreiburg) and the previous best-performing algorithm (T1-seno) across all modalities. b, Runtime comparison. c, T1-reproduce successfully replicated the performance of T1-seno across all modalities
  • Figure :