Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation

TL;DR

The paper addresses data scarcity in medical image segmentation by introducing Siamese-Diffusion, a dual-path diffusion framework that couples Mask-Diffusion (mask-only) with Image-Diffusion (mask+image) and uses a Noise Consistency Loss to steer Mask-Diffusion toward high-fidelity, morphologically rich minima. A Dense Hint Input module and Online-Augmentation further enhance priors and data volume, while sampling relies on Mask-Diffusion to maintain diversity. Across five datasets, the approach yields superior image quality (FID, KID, CLIP-I, LPIPS, CMMD, MOS) and improves segmentation performance (notably SANet by 3.6% mDice and 4.4% mIoU on Polyps; UNet by 1.52% and 1.64% on ISIC2018). The findings demonstrate that high-fidelity morphology coupled with morphological diversity enhances downstream segmentation reliability, offering a scalable path for data augmentation in medical imaging. The work provides a practical, code-supported framework for generating diverse, high-quality synthetic data to bolster segmentation models under limited annotation regimes.

Abstract

Deep learning has revolutionized medical image segmentation, yet its full potential remains constrained by the paucity of annotated datasets. While diffusion models have emerged as a promising approach for generating synthetic image-mask pairs to augment these datasets, they paradoxically suffer from the same data scarcity challenges they aim to mitigate. Traditional mask-only models frequently yield low-fidelity images due to their inability to adequately capture morphological intricacies, which can critically compromise the robustness and reliability of segmentation models. To alleviate this limitation, we introduce Siamese-Diffusion, a novel dual-component model comprising Mask-Diffusion and Image-Diffusion. During training, a Noise Consistency Loss is introduced between these components to enhance the morphological fidelity of Mask-Diffusion in the parameter space. During sampling, only Mask-Diffusion is used, ensuring diversity and scalability. Comprehensive experiments demonstrate the superiority of our method. Siamese-Diffusion boosts SANet's mDice and mIoU by 3.6% and 4.4% on the Polyps, while UNet improves by 1.52% and 1.64% on the ISIC2018. Code is available at GitHub.

Figures (12)

• Figure 1: (a) Workflow comparisons between our method and existing ones during the training and sampling phases. (b) Differences in synthesized images across methods. Mask-only methods (yellow) lack morphological characteristics (e.g., surface texture), resulting in low fidelity. Mask-image methods (red) produce high-fidelity images; however, their reliance on extra image prior control results in low diversity and scalability. Our method (blue) enhances morphological fidelity while preserving diversity.
• Figure 2: (a) Illustration of our method during the training phase. The noisy image input $z_t$ in the latent space is processed through the same diffusion model (i.e., "Copy") under the mask and image-mask conditions, generating the noise predictions $\epsilon_{\theta}^{m}$ and $\epsilon_{\theta^{\prime}}^{mix}$, respectively. These two processes are referred to as Mask-Diffusion and Image-Diffusion. The entire framework is optimized using the denoising losses from both cases and the proposed Noise Consistency Loss between $\epsilon_{\theta}^{m}$ and $\epsilon_{\theta^{\prime}}^{mix}$. The Online-Augmentation module employs single-step sampling to obtain the denoised $z_0^{\prime}$, which is then recombined with the mask $y_0$ to train the Mask-Diffusion. (b) During the sampling phase, only Mask-Diffusion is utilized to generate high-fidelity and diverse synthetic images. (c) Replacement of the Hint Input (HI) module with the proposed Dense Hint Input (DHI) module enhances the extraction of prior guidance from the image.
• Figure 3: (a) Parameter update direction. (b) Mask-Diffusion parameter update direction, scaled by the Noise Consistency Loss.
• Figure 4: t-SNE visualization of data distribution. (a)–(e) illustrate the distribution differences between real polyp images and those synthesized by each respective mask-only method. The distribution of polyp images generated by our method nearly overlaps with the real data, underscoring its exceptional ability to produce highly realistic polyp images.
• Figure 5: (a) Examples of real polyp images. (b)–(g) Examples of synthetic polyp images generated by each respective method. "M" denotes that mask-only prior control, while "M+I" denotes mask-image joint prior control. The synthetic polyp images generated by our method achieve competitive morphological fidelity while also exhibiting morphological diversity (Zoom in for better visualization).