Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UniUSNet: A Promptable Framework for Universal Ultrasound Disease Prediction and Tissue Segmentation

Zehui Lin, Zhuoneng Zhang, Xindi Hu, Zhifan Gao, Xin Yang, Yue Sun, Dong Ni, Tao Tan

TL;DR

This model handles various ultrasound types, anatomical positions, and input formats, excelling in both segmentation and classification tasks and matches state-of-the-art performance and surpasses single-dataset and ablated models.

Abstract

Ultrasound is widely used in clinical practice due to its affordability, portability, and safety. However, current AI research often overlooks combined disease prediction and tissue segmentation. We propose UniUSNet, a universal framework for ultrasound image classification and segmentation. This model handles various ultrasound types, anatomical positions, and input formats, excelling in both segmentation and classification tasks. Trained on a comprehensive dataset with over 9.7K annotations from 7 distinct anatomical positions, our model matches state-of-the-art performance and surpasses single-dataset and ablated models. Zero-shot and fine-tuning experiments show strong generalization and adaptability with minimal fine-tuning. We plan to expand our dataset and refine the prompting mechanism, with model weights and code available at (https://github.com/Zehui-Lin/UniUSNet).

UniUSNet: A Promptable Framework for Universal Ultrasound Disease Prediction and Tissue Segmentation

TL;DR

This model handles various ultrasound types, anatomical positions, and input formats, excelling in both segmentation and classification tasks and matches state-of-the-art performance and surpasses single-dataset and ablated models.

Abstract

Ultrasound is widely used in clinical practice due to its affordability, portability, and safety. However, current AI research often overlooks combined disease prediction and tissue segmentation. We propose UniUSNet, a universal framework for ultrasound image classification and segmentation. This model handles various ultrasound types, anatomical positions, and input formats, excelling in both segmentation and classification tasks. Trained on a comprehensive dataset with over 9.7K annotations from 7 distinct anatomical positions, our model matches state-of-the-art performance and surpasses single-dataset and ablated models. Zero-shot and fine-tuning experiments show strong generalization and adaptability with minimal fine-tuning. We plan to expand our dataset and refine the prompting mechanism, with model weights and code available at (https://github.com/Zehui-Lin/UniUSNet).
Paper Structure (11 sections, 1 equation, 4 figures, 3 tables)

This paper contains 11 sections, 1 equation, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Uni-Transformer Learning Framework
  4. Prompting Content
  5. Prompting Strategy
  6. Adaptive Domain Testing and Fine-tuning with Adapters
  7. Experimental Results
  8. Dataset and Implementation Details
  9. Comparison with SOTA and Ablation Study
  10. Evaluating Adapter Performance on New Datasets
  11. Conclusion

Figures (4)

  • Figure 1: The BroadUS-9.7K dataset contains 9.7K annotations of 6.9K ultrasound images from 7 different anatomical positions. (a) The number of effective instances corresponding to nature of the image, position, task and input type. Note that a breast image can contain both segmentation and classification labels, and an image with segmentation labels can form three different input types (b) The different anatomical positions, and their corresponding public dataset abbreviations.
  • Figure 2: The architecture of the proposed UniUSNet.
  • Figure 3: Some examples of segmentation result. Each column From left to right: original image, SAM, SAMUS, Single, UniUSNet w/o prompt, Prompt and ground truth.
  • Figure 4: t-SNE visualization.