Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diffusion Prism: Enhancing Diversity and Morphology Consistency in Mask-to-Image Diffusion

Hao Wang, Xiwen Chen, Ashish Bastola, Jiayou Qin, Abolfazl Razi

TL;DR

This work tackles limited diversity when generating from sparse binary masks in diffusion-based image synthesis by introducing Diffusion Prism, a training-free pixel-space augmentation that injects controlled noise and chromatic aberration to create diverse, morphologically faithful outputs. The authors analyze denoising signal transmission, show how a small artificial signal can enrich content without sacrificing structure, and validate the approach on nano-dendritic patterns against vanilla diffusion and controllable methods. Prism yields improved diversity and robust morphology preservation, with favorable CLIP alignment and nFID/SSIM metrics, and demonstrates potential as a practical data-augmentation tool for biometric patterns and other data-scarce domains. The method requires no model fine-tuning and generalizes to different biometric morphologies, offering a scalable mask-to-image solution across disciplines.

Abstract

The emergence of generative AI and controllable diffusion has made image-to-image synthesis increasingly practical and efficient. However, when input images exhibit low entropy and sparse, the inherent characteristics of diffusion models often result in limited diversity. This constraint significantly interferes with data augmentation. To address this, we propose Diffusion Prism, a training-free framework that efficiently transforms binary masks into realistic and diverse samples while preserving morphological features. We explored that a small amount of artificial noise will significantly assist the image-denoising process. To prove this novel mask-to-image concept, we use nano-dendritic patterns as an example to demonstrate the merit of our method compared to existing controllable diffusion models. Furthermore, we extend the proposed framework to other biological patterns, highlighting its potential applications across various fields.

Diffusion Prism: Enhancing Diversity and Morphology Consistency in Mask-to-Image Diffusion

TL;DR

This work tackles limited diversity when generating from sparse binary masks in diffusion-based image synthesis by introducing Diffusion Prism, a training-free pixel-space augmentation that injects controlled noise and chromatic aberration to create diverse, morphologically faithful outputs. The authors analyze denoising signal transmission, show how a small artificial signal can enrich content without sacrificing structure, and validate the approach on nano-dendritic patterns against vanilla diffusion and controllable methods. Prism yields improved diversity and robust morphology preservation, with favorable CLIP alignment and nFID/SSIM metrics, and demonstrates potential as a practical data-augmentation tool for biometric patterns and other data-scarce domains. The method requires no model fine-tuning and generalizes to different biometric morphologies, offering a scalable mask-to-image solution across disciplines.

Abstract

The emergence of generative AI and controllable diffusion has made image-to-image synthesis increasingly practical and efficient. However, when input images exhibit low entropy and sparse, the inherent characteristics of diffusion models often result in limited diversity. This constraint significantly interferes with data augmentation. To address this, we propose Diffusion Prism, a training-free framework that efficiently transforms binary masks into realistic and diverse samples while preserving morphological features. We explored that a small amount of artificial noise will significantly assist the image-denoising process. To prove this novel mask-to-image concept, we use nano-dendritic patterns as an example to demonstrate the merit of our method compared to existing controllable diffusion models. Furthermore, we extend the proposed framework to other biological patterns, highlighting its potential applications across various fields.
Paper Structure (16 sections, 10 equations, 12 figures, 2 tables)

This paper contains 16 sections, 10 equations, 12 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dendritic Patterns
  4. Image-to-Image Diffusion
  5. Methodology
  6. Signal Degradation in Denoising
  7. Initial Entropy
  8. Domain Shift in Diffusion
  9. Diffusion Prism
  10. Impact of Noise in Prism
  11. Experiments
  12. Qualitative Results
  13. Quantitative evaluation
  14. Ablation Study
  15. Discussion
  16. ...and 1 more sections

Figures (12)

  • Figure 1: Concept of the Diffusion Prism
  • Figure 2: Comparison of different diffusion frameworks. The proposed method Prism is an individual add-on module that does not require any training and does not interact with the vanilla diffusion model
  • Figure 3: Dendrite samples. Real samples (upper) are taken in lab microscopes; Artificial samples (lower) are generated using mathematical algorithms
  • Figure 4: Noise injection comparison: pixel space vs. latent space.
  • Figure 5: Noise sampling schedule in image-to-image synthesis.
  • ...and 7 more figures