Diffusion-Guided Mask-Consistent Paired Mixing for Endoscopic Image Segmentation

TL;DR

This work tackles the challenge of limited and diverse annotations in endoscopic image segmentation by uniting diffusion-based appearance diversification with mask-consistent mixing. It introduces Mask-Consistent Paired Mixing (MCPMix), which pairs real images with mask-aligned synthetic counterparts and mixes only appearance while preserving hard segmentation labels, and Real-Anchored Learnable Annealing (RLA), which adaptively reweights synthetic contributions to re-anchor learning toward real data. The approach achieves state-of-the-art performance across multiple endoscopic and dermoscopic datasets, with notable improvements in boundary accuracy and stability. By combining diffusion-driven diversity with label-preserving mixing and adaptive re-anchoring, the method offers a robust, generalizable augmentation framework for dense medical segmentation.

Abstract

Augmentation for dense prediction typically relies on either sample mixing or generative synthesis. Mixing improves robustness but misaligned masks yield soft label ambiguity. Diffusion synthesis increases apparent diversity but, when trained as common samples, overlooks the structural benefit of mask conditioning and introduces synthetic-real domain shift. We propose a paired, diffusion-guided paradigm that fuses the strengths of both. For each real image, a synthetic counterpart is generated under the same mask and the pair is used as a controllable input for Mask-Consistent Paired Mixing (MCPMix), which mixes only image appearance while supervision always uses the original hard mask. This produces a continuous family of intermediate samples that smoothly bridges synthetic and real appearances under shared geometry, enlarging diversity without compromising pixel-level semantics. To keep learning aligned with real data, Real-Anchored Learnable Annealing (RLA) adaptively adjusts the mixing strength and the loss weight of mixed samples over training, gradually re-anchoring optimization to real data and mitigating distributional bias. Across Kvasir-SEG, PICCOLO, CVC-ClinicDB, a private NPC-LES cohort, and ISIC 2017, the approach achieves state-of-the-art segmentation performance and consistent gains over baselines. The results show that combining label-preserving mixing with diffusion-driven diversity, together with adaptive re-anchoring, yields robust and generalizable endoscopic segmentation.

Figures (7)

• Figure 1: Comparison of augmentation strategies. (a) Inter-sample mixing via linear mixing or copy-and-paste introduces semantic ambiguities and disrupted anatomical continuity. (b) Generative augmentation adds diversity, while underuses same-mask, and induces domain shift. (c) We mix each real image with a mask-consistent synthetic counterpart via diffusion model to preserve lesion geometry and train with hard label supervision, thereby boosting segmentation accuracy and robustness.
• Figure 2: The proposed three-stage pipeline. Stage 1: a frozen depth network provides priors. Stage 2: diffusion-guided ControlNet is trained with mask, depth, and text, then frozen. Stage 3: MCPMix aims to preserve semantics and boundaries while expanding appearance diversity. RLA adaptively reduces both mixing strength and the loss weight of mixed samples, transitioning from strong exploration to real-domain.
• Figure 3: Visualization of different methods on Kvasir-SEG, CVC ClinicDB, PICCOLO and NPC-LES. (A:Image, B:Ground-Truth, C:Ours, D:CutMix, E:Mixup, F:GridMix, G:SmoothMix, H:PuzzleMix, I:Augmix, J:PixMix, K:DiffuseMix, L:HSMix)
• Figure 4: Visualization of different methods on ISIC 2017 dataset.
• Figure 5: Distribution changes during training process.