Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AMA-SAM: Adversarial Multi-Domain Alignment of Segment Anything Model for High-Fidelity Histology Nuclei Segmentation

Jiahe Qian, Yaoyu Fang, Jinkui Hao, Bo Zhou

TL;DR

AMA-SAM tackles histology nuclei segmentation by extending SAM with a Conditional Gradient Reversal Layer for selective multi-domain alignment and a High-Resolution Decoder to produce $1024 \times 1024$ segmentations. The method freezes SAM while training added components, enabling domain-invariant representations that still preserve primary-domain features. Across four public datasets, AMA-SAM consistently improves segmentation accuracy on the primary dataset, outperforming state-of-the-art single- and multi-domain baselines and demonstrating robust boundary delineation. This approach unlocks high-fidelity, cross-domain nuclei segmentation with practical implications for digital pathology analysis.

Abstract

Accurate segmentation of cell nuclei in histopathology images is essential for numerous biomedical research and clinical applications. However, existing cell nucleus segmentation methods only consider a single dataset (i.e., primary domain), while neglecting to leverage supplementary data from diverse sources (i.e., auxiliary domains) to reduce overfitting and enhance the performance. Although incorporating multiple datasets could alleviate overfitting, it often exacerbates performance drops caused by domain shifts. In this work, we introduce Adversarial Multi-domain Alignment of Segment Anything Model (AMA-SAM) that extends the Segment Anything Model (SAM) to overcome these obstacles through two key innovations. First, we propose a Conditional Gradient Reversal Layer (CGRL), a multi-domain alignment module that harmonizes features from diverse domains to promote domain-invariant representation learning while preserving crucial discriminative features for the primary dataset. Second, we address SAM's inherent low-resolution output by designing a High-Resolution Decoder (HR-Decoder), which directly produces fine-grained segmentation maps in order to capture intricate nuclei boundaries in high-resolution histology images. To the best of our knowledge, this is the first attempt to adapt SAM for multi-dataset learning with application to histology nuclei segmentation. We validate our method on several publicly available datasets, demonstrating consistent and significant improvements over state-of-the-art approaches.

AMA-SAM: Adversarial Multi-Domain Alignment of Segment Anything Model for High-Fidelity Histology Nuclei Segmentation

TL;DR

AMA-SAM tackles histology nuclei segmentation by extending SAM with a Conditional Gradient Reversal Layer for selective multi-domain alignment and a High-Resolution Decoder to produce segmentations. The method freezes SAM while training added components, enabling domain-invariant representations that still preserve primary-domain features. Across four public datasets, AMA-SAM consistently improves segmentation accuracy on the primary dataset, outperforming state-of-the-art single- and multi-domain baselines and demonstrating robust boundary delineation. This approach unlocks high-fidelity, cross-domain nuclei segmentation with practical implications for digital pathology analysis.

Abstract

Accurate segmentation of cell nuclei in histopathology images is essential for numerous biomedical research and clinical applications. However, existing cell nucleus segmentation methods only consider a single dataset (i.e., primary domain), while neglecting to leverage supplementary data from diverse sources (i.e., auxiliary domains) to reduce overfitting and enhance the performance. Although incorporating multiple datasets could alleviate overfitting, it often exacerbates performance drops caused by domain shifts. In this work, we introduce Adversarial Multi-domain Alignment of Segment Anything Model (AMA-SAM) that extends the Segment Anything Model (SAM) to overcome these obstacles through two key innovations. First, we propose a Conditional Gradient Reversal Layer (CGRL), a multi-domain alignment module that harmonizes features from diverse domains to promote domain-invariant representation learning while preserving crucial discriminative features for the primary dataset. Second, we address SAM's inherent low-resolution output by designing a High-Resolution Decoder (HR-Decoder), which directly produces fine-grained segmentation maps in order to capture intricate nuclei boundaries in high-resolution histology images. To the best of our knowledge, this is the first attempt to adapt SAM for multi-dataset learning with application to histology nuclei segmentation. We validate our method on several publicly available datasets, demonstrating consistent and significant improvements over state-of-the-art approaches.

Paper Structure

This paper contains 13 sections, 6 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Overall Pipeline of AMA-SAM
  5. Conditional Gradient Reversal Layer
  6. High-Resolution Decoder
  7. Objective Functions
  8. Implementation Details
  9. Datasets, Baselines, and Evaluation Metrics
  10. Experimental Results
  11. Main Results
  12. Ablation Studies
  13. Conclusion

Figures (2)

  • Figure 1: Flowchart for training and inference of AMA-SAM and its crucial components. Our AMA-SAM is the first model which can utilize auxiliary datasets with distinct domain distributions to enhance the nuclei segmentation performance on the primary dataset.
  • Figure 2: Visualization comparison of nuclei segmentation results for U-Net (3rd column), UN-SAM (4th column), and AMA-SAM (5th column) on MoNuSeg and TNBC datasets. The corresponding segmentation errors are reported on the right with the Dice score computed between the predictions and the human label (2nd column).