PathSeqSAM: Sequential Modeling for Pathology Image Segmentation with SAM2

TL;DR

This work tackles cross-slice information in histopathology segmentation by treating individual 2D slices from the same subject as a sequential sequence processed with SAM2's memory. Key innovations include a distance-aware cross-slice attention mechanism, an adaptive memory selection strategy, and LoRA-based domain adaptation to pathology. On KPI Challenge 2024 glomeruli segmentation, PathSeqSAM achieves a mean Dice of 94.71 with a 2.23% improvement over SAM2 and outperforms baseline methods like nnUNet and Swin-Unet. The approach improves robustness to staining variability and nonuniform slice spacing, and the authors release public code.

Abstract

Current methods for pathology image segmentation typically treat 2D slices independently, ignoring valuable cross-slice information. We present PathSeqSAM, a novel approach that treats 2D pathology slices as sequential video frames using SAM2's memory mechanisms. Our method introduces a distance-aware attention mechanism that accounts for variable physical distances between slices and employs LoRA for domain adaptation. Evaluated on the KPI Challenge 2024 dataset for glomeruli segmentation, PathSeqSAM demonstrates improved segmentation quality, particularly in challenging cases that benefit from cross-slice context. We have publicly released our code at https://github.com/JackyyyWang/PathSeqSAM.

Figures (1)

• Figure 1: Proposed SAM2 segmentation pipeline for pathology images. Features extracted via the LoRA-adapted SAM2 encoder are refined by distance-aware cross-slice attention and a memory bank, enabling coherent segmentation across slices.