Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RAP: Retrieve, Adapt, and Prompt-Fit for Training-Free Few-Shot Medical Image Segmentation

Zhihao Mao, Bangpu Chen

Abstract

Few-shot medical image segmentation (FSMIS) has achieved notable progress, yet most existing methods mainly rely on semantic correspondences from scarce annotations while under-utilizing a key property of medical imagery: anatomical targets exhibit repeatable high-frequency morphology (e.g., boundary geometry and spatial layout) across patients and acquisitions. We propose RAP, a training-free framework that retrieves, adapts, and prompts Segment Anything Model 2 (SAM2) for FSMIS. First, RAP retrieves morphologically compatible supports from an archive using DINOv3 features to reduce brittleness in single-support choice. Second, it adapts the retrieved support mask to the query by fitting boundary-aware structural cues, yielding an anatomy-consistent pre-mask under domain shifts. Third, RAP converts the pre-mask into prompts by sampling positive points via Voronoi partitioning and negative points via sector-based sampling, and feeds them into SAM2 for final refinement without any fine-tuning. Extensive experiments on multiple medical segmentation benchmarks show that RAP consistently surpasses prior FSMIS baselines and achieves state-of-the-art performance. Overall, RAP demonstrates that explicit structural fitting combined with retrieval-augmented prompting offers a simple and effective route to robust training-free few-shot medical segmentation.

RAP: Retrieve, Adapt, and Prompt-Fit for Training-Free Few-Shot Medical Image Segmentation

Abstract

Few-shot medical image segmentation (FSMIS) has achieved notable progress, yet most existing methods mainly rely on semantic correspondences from scarce annotations while under-utilizing a key property of medical imagery: anatomical targets exhibit repeatable high-frequency morphology (e.g., boundary geometry and spatial layout) across patients and acquisitions. We propose RAP, a training-free framework that retrieves, adapts, and prompts Segment Anything Model 2 (SAM2) for FSMIS. First, RAP retrieves morphologically compatible supports from an archive using DINOv3 features to reduce brittleness in single-support choice. Second, it adapts the retrieved support mask to the query by fitting boundary-aware structural cues, yielding an anatomy-consistent pre-mask under domain shifts. Third, RAP converts the pre-mask into prompts by sampling positive points via Voronoi partitioning and negative points via sector-based sampling, and feeds them into SAM2 for final refinement without any fine-tuning. Extensive experiments on multiple medical segmentation benchmarks show that RAP consistently surpasses prior FSMIS baselines and achieves state-of-the-art performance. Overall, RAP demonstrates that explicit structural fitting combined with retrieval-augmented prompting offers a simple and effective route to robust training-free few-shot medical segmentation.

Paper Structure

This paper contains 28 sections, 14 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary
  4. Problem Formulation
  5. Notations
  6. Foundation Models
  7. Methodology
  8. Overview
  9. Retrieve Stage
  10. Adapt Stage
  11. Frequency-domain Style Adaptation
  12. DINO-based Semantic Gating
  13. Oriented Chamfer Matching
  14. Prompt-Fit Stage
  15. Voronoi-based Positive Points
  16. ...and 13 more sections

Figures (3)

  • Figure 1: Overview of the proposed RAP framework. Given a query image, we first retrieve the most similar support from a database using DINOv3 features (Retrieve). Then, we adapt the support shape to the query through oriented chamfer matching with semantic gating (Adapt). Finally, we generate Voronoi-based point prompts and feed them to SAM2 for segmentation (Prompt-Fit).
  • Figure 2: Frequency-domain style adaptation via wavelet transform. The support's low-frequency (LL) subband is replaced with the query's LL to align global appearance while preserving high-frequency edge details.
  • Figure 3: Qualitative results on Card-MRI for three cardiac structures. Our method produces more accurate boundaries, especially for the thin myocardium (LV-MYO).