Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Structure-Guided Histopathology Synthesis via Dual-LoRA Diffusion

Xuan Xu, Prateek Prasanna

TL;DR

This work proposes Dual-LoRA Controllable Diffusion, a unified centroid-guided diffusion framework that jointly supports Local Structure Completion and Global Structure Synthesis within a single model, supporting scalable pan-cancer histopathology modeling.

Abstract

Histopathology image synthesis plays an important role in tissue restoration, data augmentation, and modeling of tumor microenvironments. However, existing generative methods typically address restoration and generation as separate tasks, although both share the same objective of structure-consistent tissue synthesis under varying degrees of missingness, and often rely on weak or inconsistent structural priors that limit realistic cellular organization. We propose Dual-LoRA Controllable Diffusion, a unified centroid-guided diffusion framework that jointly supports Local Structure Completion and Global Structure Synthesis within a single model. Multi-class nuclei centroids serve as lightweight and annotation-efficient spatial priors, providing biologically meaningful guidance under both partial and complete image absence. Two task-specific LoRA adapters specialize the shared backbone for local and global objectives without retraining separate diffusion models. Extensive experiments demonstrate consistent improvements over state-of-the-art GAN and diffusion baselines across restoration and synthesis tasks. For local completion, LPIPS computed within the masked region improves from 0.1797 (HARP) to 0.1524, and for global synthesis, FID improves from 225.15 (CoSys) to 76.04, indicating improved structural fidelity and realism. Our approach achieves more faithful structural recovery in masked regions and substantially improved realism and morphology consistency in full synthesis, supporting scalable pan-cancer histopathology modeling.

Structure-Guided Histopathology Synthesis via Dual-LoRA Diffusion

TL;DR

This work proposes Dual-LoRA Controllable Diffusion, a unified centroid-guided diffusion framework that jointly supports Local Structure Completion and Global Structure Synthesis within a single model, supporting scalable pan-cancer histopathology modeling.

Abstract

Histopathology image synthesis plays an important role in tissue restoration, data augmentation, and modeling of tumor microenvironments. However, existing generative methods typically address restoration and generation as separate tasks, although both share the same objective of structure-consistent tissue synthesis under varying degrees of missingness, and often rely on weak or inconsistent structural priors that limit realistic cellular organization. We propose Dual-LoRA Controllable Diffusion, a unified centroid-guided diffusion framework that jointly supports Local Structure Completion and Global Structure Synthesis within a single model. Multi-class nuclei centroids serve as lightweight and annotation-efficient spatial priors, providing biologically meaningful guidance under both partial and complete image absence. Two task-specific LoRA adapters specialize the shared backbone for local and global objectives without retraining separate diffusion models. Extensive experiments demonstrate consistent improvements over state-of-the-art GAN and diffusion baselines across restoration and synthesis tasks. For local completion, LPIPS computed within the masked region improves from 0.1797 (HARP) to 0.1524, and for global synthesis, FID improves from 225.15 (CoSys) to 76.04, indicating improved structural fidelity and realism. Our approach achieves more faithful structural recovery in masked regions and substantially improved realism and morphology consistency in full synthesis, supporting scalable pan-cancer histopathology modeling.
Paper Structure (22 sections, 10 equations, 4 figures, 3 tables)

This paper contains 22 sections, 10 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Unified Problem Formulation
  4. Dual-LoRA Specialization
  5. Local Structure Completion
  6. Mask-aware diffusion loss.
  7. Image-domain refinement.
  8. Final objective.
  9. Global Structure Synthesis
  10. Centroid-aware diffusion loss.
  11. Layout regularization.
  12. Final objective.
  13. Experiments and Results
  14. Dataset
  15. Implementation Details
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Challenges in histopathology local structure completion with complex masks. Pix2Pix isola2017image and HARP harp fail to reconstruct fine-grained structures under large irregular masks, while our method preserves coherent morphology and tissue continuity. Red boxes highlight reconstruction failures in the baselines.
  • Figure 2: Overview of Dual-LoRA Controllable Diffusion. Given image $x$, mask $m$, and centroid maps $C$, we build a unified spatial condition. For inpainting, $[x_{\mathrm{hint}}, m, C]$ with $x_{\mathrm{hint}}=x\odot(1-m)$; for generation, $[\mathbf{0}, \mathbf{0}, C]$. Stable Diffusion and ControlNet are frozen, while two LoRA adapters specialize the two modes.
  • Figure 3: Qualitative comparison on inpainting under centroid-guided control.
  • Figure 4: Qualitative evaluation of centroid-guided global structure synthesis. (a) Under centroid guidance, Dual-LoRA produces coherent H&E morphology compared to Pix2Pix and CoSys. (b) Downstream cancer-type discrimination (LIHC, MESO) shows preservation of class-specific morphology.