Reducing Annotation Burden: Exploiting Image Knowledge for Few-Shot Medical Video Object Segmentation via Spatiotemporal Consistency Relearning

TL;DR

This work tackles the high cost of pixel-level annotations for medical video segmentation by proposing a two-phase framework that leverages abundant labeled images and very sparse video labels. The approach first trains a few-shot segmentation model on images, then performs spatiotemporal consistency relearning on medical videos, enforcing temporal continuity and cross-model (image vs. relearning) consistency at both feature and prediction levels. Key contributions include a pseudo mask-guided cross-resolution fusion for images, a temporal attention-based relearning module, and a set of consistency losses that bridge image and video domains under an extremely low-data regime, achieving state-of-the-art performance on medical video benchmarks. The method significantly reduces annotation burdens while delivering accurate video segmentation, with practical implications for clinical workflows and AI-assisted diagnostics; code is released at the provided repository.

Abstract

Few-shot video object segmentation aims to reduce annotation costs; however, existing methods still require abundant dense frame annotations for training, which are scarce in the medical domain. We investigate an extremely low-data regime that utilizes annotations from only a few video frames and leverages existing labeled images to minimize costly video annotations. Specifically, we propose a two-phase framework. First, we learn a few-shot segmentation model using labeled images. Subsequently, to improve performance without full supervision, we introduce a spatiotemporal consistency relearning approach on medical videos that enforces consistency between consecutive frames. Constraints are also enforced between the image model and relearning model at both feature and prediction levels. Experiments demonstrate the superiority of our approach over state-of-the-art few-shot segmentation methods. Our model bridges the gap between abundant annotated medical images and scarce, sparsely labeled medical videos to achieve strong video segmentation performance in this low data regime. Code is available at https://github.com/MedAITech/RAB.

Figures (2)

• Figure 1: Pipeline of the proposed few-shot segmentation method for medical video object segmentation.
• Figure 2: Qualitative results of our few-shot video object segmentation model on the HMC-QU and ASU-Mayo datasets. The first column shows annotated support frames with ground truth masks (yellow). The remaining columns illustrate our model's segmentation predictions (green masks) on sampled query frames from videos. Ground truth masks for the query frames are outlined in white for reference.