Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SAMPO-Path: Segmentation Intent-Aligned Preference Optimization for Pathology Foundation Model Segmentation

Yonghuang Wu, Wenwen Zeng, Xuan Xie, Chengqian Zhao, Guoqing Wu, Jinhua Yu

Abstract

Foundation models have shown strong performance in multi-object segmentation with visual prompts, yet histopathology images remain challenging due to high cellular density, heterogeneity, and the gap between pixel-level supervision and clinical segmentation intent (e.g., selectively segmenting nuclei of a specific type). In practice, such intents are expressed through diverse and noisy prompts, causing prompt-intent misalignment and inconsistent predictions. We introduce SAMPO (Segmentation Anything Model with Preference Optimization), a preference-aligned fine-tuning framework that explicitly aligns pathology foundation models with clinical segmentation intent. SAMPO is the first to adapt Direct Preference Optimization (DPO) to pure vision foundation models, enabling accurate segmentation from minimal and imperfect prompts. The framework features three key components: (1) online prompt-centric preference mining to synthesize preference pairs across prompt qualities; (2) multi-mask preference learning to leverage output ambiguity for fine-grained ranking supervision; and (3) a hybrid loss combining preference optimization with pixel-level supervision for stable training. Trained on two datasets covering four tasks and evaluated on corresponding test sets and 12 external validation datasets, SAMPO consistently improves segmentation accuracy, robustness to prompt variations, and clinical intent adherence in dense histopathology images.

SAMPO-Path: Segmentation Intent-Aligned Preference Optimization for Pathology Foundation Model Segmentation

Abstract

Foundation models have shown strong performance in multi-object segmentation with visual prompts, yet histopathology images remain challenging due to high cellular density, heterogeneity, and the gap between pixel-level supervision and clinical segmentation intent (e.g., selectively segmenting nuclei of a specific type). In practice, such intents are expressed through diverse and noisy prompts, causing prompt-intent misalignment and inconsistent predictions. We introduce SAMPO (Segmentation Anything Model with Preference Optimization), a preference-aligned fine-tuning framework that explicitly aligns pathology foundation models with clinical segmentation intent. SAMPO is the first to adapt Direct Preference Optimization (DPO) to pure vision foundation models, enabling accurate segmentation from minimal and imperfect prompts. The framework features three key components: (1) online prompt-centric preference mining to synthesize preference pairs across prompt qualities; (2) multi-mask preference learning to leverage output ambiguity for fine-grained ranking supervision; and (3) a hybrid loss combining preference optimization with pixel-level supervision for stable training. Trained on two datasets covering four tasks and evaluated on corresponding test sets and 12 external validation datasets, SAMPO consistently improves segmentation accuracy, robustness to prompt variations, and clinical intent adherence in dense histopathology images.

Paper Structure

This paper contains 26 sections, 6 equations, 8 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Medical Visual Foundation Model
  4. Preference Learning for Model Alignment
  5. Methodology
  6. Preliminaries and Motivation
  7. Clinical Segmentation Intent in Pathology
  8. Preference Optimization for Intent Alignment
  9. The SAMPO Framework
  10. Online Prompt-Centric Preference Mining
  11. Fine-Grained Learning via Multi-Mask Ambiguity
  12. Preference Optimization Objective for Pathology
  13. Hybrid Optimization for Clinical Stability
  14. Experiments
  15. Dataset and Evaluation
  16. ...and 11 more sections

Figures (8)

  • Figure 1: Illustration of preference-optimized segmentation versus traditional pixel-centric paradigms.
  • Figure 2: Overview.
  • Figure 3: Performance trends of different segmentation methods on PanNuke-T1 and PanNuke-T2 under varying amounts of training data. Metric: IoU.
  • Figure 4: Point prompt sensitivity analysis on the PanNuke T1 and T2. Heatmaps show the Dice scores of SAMPO and the original SAM2 across varying numbers of positive (1–10) and negative (0–10) point prompts. A unified color scale is used for direct comparison.
  • Figure 5: The qualitative results in the PanNuke dataset. The arrow indicates the key differences.
  • ...and 3 more figures