Is Foreground Prototype Sufficient? Few-Shot Medical Image Segmentation with Background-Fused Prototype

TL;DR

The paper tackles the problem of few-shot medical image segmentation where foreground-only prototypes struggle due to concentrated frequency content and overlapping background features. It introduces Background-fused Prototype (Bro), a plug-in framework that combines Feature Similarity Calibration (FeaC) and Hierarchical Channel-Adversarial Attention (HiCA) to fuse background information into segmentation prototypes within a $N$-way $K$-shot meta-learning setting, optimized with $\\mathcal{L}_{seg}$, $\\mathcal{L}_{reg}$, and $\\beta \\\mathcal{L}_{adv}$. Empirical results on ABD-CT, ABD-MRI, and CMR demonstrate consistent Dice-score gains over state-of-the-art prototypical FSS methods, validating the importance of explicit background modeling in medical images. The work provides a practical, plug-in improvement for existing FSS models, enabling more accurate clinical segmentation with limited annotations and offering insights into background representation analysis through both qualitative and quantitative evaluations.

Abstract

Few-shot Semantic Segmentation(FSS)aim to adapt a pre-trained model to new classes with as few as a single labeled training sample per class. The existing prototypical work used in natural image scenarios biasedly focus on capturing foreground's discrimination while employing a simplistic representation for background, grounded on the inherent observation separation between foreground and background. However, this paradigm is not applicable to medical images where the foreground and background share numerous visual features, necessitating a more detailed description for background. In this paper, we present a new pluggable Background-fused prototype(Bro)approach for FSS in medical images. Instead of finding a commonality of background subjects in support image, Bro incorporates this background with two pivot designs. Specifically, Feature Similarity Calibration(FeaC)initially reduces noise in the support image by employing feature cross-attention with the query image. Subsequently, Hierarchical Channel Adversarial Attention(HiCA)merges the background into comprehensive prototypes. We achieve this by a channel groups-based attention mechanism, where an adversarial Mean-Offset structure encourages a coarse-to-fine fusion. Extensive experiments show that previous state-of-the-art methods, when paired with Bro, experience significant performance improvements. This demonstrates a more integrated way to represent backgrounds specifically for medical image.

Figures (9)

• Figure 1: Motivation of Bro. (a) As shown in the comparison of probability distribution of frequency spectrum entropy (the experiment is elaborated in Supplementary), the lower mean of medical images suggests a more concentrated frequency distribution than natural images. Correspondingly, the background in medical images has more similar features to the foreground, necessitating a further background representation to discriminate it from the foreground. To this end, we propose a background fusion scheme Bro whose idea is illustrated in (b) intuitively.
• Figure 2: Overview of the SSL-ALPNet framework plugged with Bro. (a) Unlike directly trimming background prototypes in the conventional pipeline (marked by gray lines), Bro provides an ability of discriminative background representation. In this module, (b) FeaC denoise support feature map ${F}_s$ by calibrating similarity with query feature map ${F}_q$. After that, (c) HiCA generates detailed background representation $\widetilde{F}_S$ by performing a channel group attention-based fusion over the similarity calibrated $\hat{F}_S$.
• Figure 3: The qualitative comparison results on ABD-MRI (the left side) and ABD-CT (the right side) under Setting-2. Top to bottom: Support images, segmentation results and ground-truth segmentation of a query slice containing the target object (Best viewed with zoom).
• Figure 4: The qualitative comparison results in CMR under Setting-1. Left to right: Support images, segmentation results and ground-truth segmentation of a query slice containing the target object. Top to bottom: LV-MYO (left ventricular myocardium), RV (right ventricle) and LV-BP (left ventricular outflow tract blood pool). (Best viewed with zoom)
• Figure 5: Background representation analysis on ABD-CT in Setting-1. Left: Example image with foreground marked in purple; Middle: Variation of convolutional similarity between the background-fused prototypes and the foreground over epoch 3$\sim$100 with gap 25; Right: Details as epoch varies from 3 to 25.