Unpaired Multi-Domain Histopathology Virtual Staining using Dual Path Prompted Inversion

TL;DR

This work tackles the challenge of unpaired multi-domain virtual staining in histopathology by introducing a dual-path prompted inversion framework that leverages a single pretrained diffusion model. Structural fidelity is maintained via a Structural Target Path with StainStructPrompt, while style transfer is controlled through a Style Target Path with StainStylePrompt, enabling complete content–style disentanglement without model fine-tuning. The method demonstrates state-of-the-art performance on unpaired multi-domain staining tasks (H&E→MAS/PAS) with robust ablations showing the benefits of both prompts and their interaction, validated on the ANHIR dataset. The approach offers a training-free, controllable, and scalable solution for clinical-stain translation that preserves diagnostic content and could streamline multi-stain workflows.

Abstract

Virtual staining leverages computer-aided techniques to transfer the style of histochemically stained tissue samples to other staining types. In virtual staining of pathological images, maintaining strict structural consistency is crucial, as these images emphasize structural integrity more than natural images. Even slight structural alterations can lead to deviations in diagnostic semantic information. Furthermore, the unpaired characteristic of virtual staining data may compromise the preservation of pathological diagnostic content. To address these challenges, we propose a dual-path inversion virtual staining method using prompt learning, which optimizes visual prompts to control content and style, while preserving complete pathological diagnostic content. Our proposed inversion technique comprises two key components: (1) Dual Path Prompted Strategy, we utilize a feature adapter function to generate reference images for inversion, providing style templates for input image inversion, called Style Target Path. We utilize the inversion of the input image as the Structural Target path, employing visual prompt images to maintain structural consistency in this path while preserving style information from the style Target path. During the deterministic sampling process, we achieve complete content-style disentanglement through a plug-and-play embedding visual prompt approach. (2) StainPrompt Optimization, where we only optimize the null visual prompt as ``operator'' for dual path inversion, rather than fine-tune pre-trained model. We optimize null visual prompt for structual and style trajectory around pivotal noise on each timestep, ensuring accurate dual-path inversion reconstruction. Extensive evaluations on publicly available multi-domain unpaired staining datasets demonstrate high structural consistency and accurate style transfer results.

Figures (10)

• Figure 1: Structural consistency is crucial in virtual staining. a) In natural image translation, changes in non-core details do not affect the semantic content of the core. b) In pathological images, every detail holds diagnostic significance, and even minor changes can alter the diagnostic information.
• Figure 2: The overall framework of the proposed dual path inversion method. By optimize null-visual prompt to correct deviation of deterministic sampling, Pre-trained diffusion models can maintain a high degree of structural consistency. Meanwhile, the proposed StainStyleControl leverage a constant to control the degree of influence of style trajectory.
• Figure 3: The virtual generation of MAS, PAS, and PASM stained images from H&E stained images using our network.
• Figure 4: Comparison of different methods of staining migration from the same H&E staining image to MAS staining. Our model performs state-of-art in various indicators and subtle structures.
• Figure 5: Quantitative comparison with StainStructPrompt: (a) Input H&E image, (b) unconditional inversion without prompt, (c) unconditional inversion with prompt, (d) conditional inversion without prompt, (e) conditional inversion with prompt. Our structural prompt optimization tunes structural information of the deviation into the prompt.