Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

StainFuser: Controlling Diffusion for Faster Neural Style Transfer in Multi-Gigapixel Histology Images

Robert Jewsbury, Ruoyu Wang, Abhir Bhalerao, Nasir Rajpoot, Quoc Dang Vu

TL;DR

StainFuser tackles stain normalization under domain shift in histology by reframing the task as neural style transfer using a conditional latent diffusion model. It introduces SPI-2M, a dataset with over 2 million NST-generated image pairs, to train a diffusion-based StainFuser that preserves morphology while applying target stain characteristics. Across CoNIC and Atypia-14 assessments, StainFuser outperforms handcrafted, GAN-based, and prior diffusion-based methods in image quality and downstream nuclei segmentation/classification, while delivering roughly 30× faster inference than NST. The work demonstrates strong consistency on whole-slide images and highlights practical considerations, including resolution choice, denoising steps, and data volume, for deploying diffusion-based stain normalization in clinical pathology workflows.

Abstract

Stain normalization algorithms aim to transform the color and intensity characteristics of a source multi-gigapixel histology image to match those of a target image, mitigating inconsistencies in the appearance of stains used to highlight cellular components in the images. We propose a new approach, StainFuser, which treats this problem as a style transfer task using a novel Conditional Latent Diffusion architecture, eliminating the need for handcrafted color components. With this method, we curate SPI-2M the largest stain normalization dataset to date of over 2 million histology images with neural style transfer for high-quality transformations. Trained on this data, StainFuser outperforms current state-of-the-art deep learning and handcrafted methods in terms of the quality of normalized images and in terms of downstream model performance on the CoNIC dataset.

StainFuser: Controlling Diffusion for Faster Neural Style Transfer in Multi-Gigapixel Histology Images

TL;DR

StainFuser tackles stain normalization under domain shift in histology by reframing the task as neural style transfer using a conditional latent diffusion model. It introduces SPI-2M, a dataset with over 2 million NST-generated image pairs, to train a diffusion-based StainFuser that preserves morphology while applying target stain characteristics. Across CoNIC and Atypia-14 assessments, StainFuser outperforms handcrafted, GAN-based, and prior diffusion-based methods in image quality and downstream nuclei segmentation/classification, while delivering roughly 30× faster inference than NST. The work demonstrates strong consistency on whole-slide images and highlights practical considerations, including resolution choice, denoising steps, and data volume, for deploying diffusion-based stain normalization in clinical pathology workflows.

Abstract

Stain normalization algorithms aim to transform the color and intensity characteristics of a source multi-gigapixel histology image to match those of a target image, mitigating inconsistencies in the appearance of stains used to highlight cellular components in the images. We propose a new approach, StainFuser, which treats this problem as a style transfer task using a novel Conditional Latent Diffusion architecture, eliminating the need for handcrafted color components. With this method, we curate SPI-2M the largest stain normalization dataset to date of over 2 million histology images with neural style transfer for high-quality transformations. Trained on this data, StainFuser outperforms current state-of-the-art deep learning and handcrafted methods in terms of the quality of normalized images and in terms of downstream model performance on the CoNIC dataset.
Paper Structure (35 sections, 6 equations, 18 figures, 7 tables)

This paper contains 35 sections, 6 equations, 18 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Creating SPI-2M
  5. Slide Selection
  6. Patch selection.
  7. Source and target selection.
  8. Neural Style Transfer
  9. Generating the transferred set.
  10. Design of StainFuser
  11. Latent Diffusion Models
  12. StainFuser Architecture
  13. Experiments
  14. Evalution Datasets
  15. Experimental Settings
  16. ...and 20 more sections

Figures (18)

  • Figure 1: The diagram of the proposed StainFuser. StainFuser takes in a source and target image to predict the stain normalized version of the source image. The application of StainFuser was demonstrated through a nuclei segmentation and classification task and a WSI-level inference task.
  • Figure 2: Overview of the data curation workflow: Slides were sourced from the TCGA repository, followed by the patch extraction from identified tissue regions. A two-stage clustering pipeline was implemented to select biologically meaningful and representative patches, ensuring an accurate representation of the real-world morphology and color distribution.
  • Figure 3: Qualitative comparisons between StainFuser and other methods on CoNIC test set examples. All inference was performed at $512^2$ resolution and then resized for display purposes. Only StainFuser and NST preserve the color contrast between important tissue components such as stroma, glands, lumen and blood vessels present in the original image.
  • Figure 4: Target images selected by sampling in HSV space and a test sample normalised by each method assessed. Targets are displayed on 2D plane where x-axis is Hue and y-axis is Saturation by the mean value of the respective target's Hue and Saturation. High-resolution versions of each set of images are included in the \ref{['sec:addResults']}.
  • Figure 5: Heatmaps of the difference in the $m\mathcal{PQ^+} AUC$ between the Control and the test set where its color was shifted w.r.t each sampled target. Changes in performance are displayed in the same pattern as their corresponding target in \ref{['fig:qualOverallMacro']}. CAGAN is excluded as it can not normalize w.r.t. a specific target image.
  • ...and 13 more figures