SAMSA 2.0: Prompting Segment Anything with Spectral Angles for Hyperspectral Interactive Medical Image Segmentation

TL;DR

The paper addresses hyperspectral medical image segmentation where annotation is scarce and hardware-induced variability complicates learning. It introduces SAMSA 2.0, which embeds spectral angle similarity as an input prompt to achieve early spectral–spatial fusion within the Segment Anything Model, enabling robust interactive segmentation without retraining. Results show SAMSA 2.0 yields Dice gains of up to 3.8 percentage points over RGB baselines and 3.1 points over prior spectral fusion methods, while supporting few-shot and zero-shot generalization. The work also adds automatic proposal generation and analyzes mixed-training effects, highlighting practical pathways for clinical adoption with lightweight post-processing. Overall, SAMSA 2.0 offers a data-efficient, interpretable approach to hyperspectral tissue segmentation that can aid intraoperative decision-making with minimal clinician input.

Abstract

We present SAMSA 2.0, an interactive segmentation framework for hyperspectral medical imaging that introduces spectral angle prompting to guide the Segment Anything Model (SAM) using spectral similarity alongside spatial cues. This early fusion of spectral information enables more accurate and robust segmentation across diverse spectral datasets. Without retraining, SAMSA 2.0 achieves up to +3.8% higher Dice scores compared to RGB-only models and up to +3.1% over prior spectral fusion methods. Our approach enhances few-shot and zero-shot performance, demonstrating strong generalization in challenging low-data and noisy scenarios common in clinical imaging.

Figures (8)

• Figure 1: SAMSA 2.0 Architecture featuring the early fusion of spectral similarity maps into the prompt encoder.
• Figure 2: Sample segmentation results on the HIB dataset, tumour class. A) Pseudo-RGB image of the target. B) SAMSA2-Large standard predicted similarity. C) Ground truth labels. D) SAM2-Large-FT using SAM2 predicted similarity
• Figure 3: Sample segmentation results on the SB-X dataset, tumour class. A) Pseudo-RGB image of the target. B) SAMSA2-Large standard predicted similarity. C) Ground truth labels. D) SAM2-Large-FT using SAM2 predicted similarity
• Figure 4: Sample segmentation results on the SB-H dataset, tumour class. A) Pseudo-RGB image of the target. B) SAMSA2-Large standard predicted similarity. C) Ground truth labels. D) SAM2-Large-FT using SAM2 predicted similarity
• Figure 5: Sample segmentation results on the HEIPOR dataset, liver class. A) Pseudo-RGB image of the target. B) SAMSA2-Large standard predicted similarity. C) Ground truth labels. D) SAM2-Large-FT using SAM2 predicted similarity