Beyond Pixel Simulation: Pathology Image Generation via Diagnostic Semantic Tokens and Prototype Control

TL;DR

UniPath addresses the gap between diagnostic understanding and pixel-based pathology generation by unifying diagnosis-aware semantics with controllable synthesis through Multi-Stream Control. It leverages a frozen pathology MLLM to distill Diagnostic Semantic Tokens and a non-parametric Prototype Bank for component-level morphology, guided by a Flow-Matching diffusion backbone. The authors curate a 2.65M image–text corpus and a 68K high-quality subset, establishing a four-tier evaluation that demonstrates state-of-the-art visual fidelity, robust text–image alignment, and strong fine-grained controllability, with meaningful downstream augmentation benefits. The work holds significant practical impact for data augmentation, education, and research in computational pathology, while outlining ethical considerations and future directions toward higher resolution and editable pathology synthesis.

Abstract

In computational pathology, understanding and generation have evolved along disparate paths: advanced understanding models already exhibit diagnostic-level competence, whereas generative models largely simulate pixels. Progress remains hindered by three coupled factors: the scarcity of large, high-quality image-text corpora; the lack of precise, fine-grained semantic control, which forces reliance on non-semantic cues; and terminological heterogeneity, where diverse phrasings for the same diagnostic concept impede reliable text conditioning. We introduce UniPath, a semantics-driven pathology image generation framework that leverages mature diagnostic understanding to enable controllable generation. UniPath implements Multi-Stream Control: a Raw-Text stream; a High-Level Semantics stream that uses learnable queries to a frozen pathology MLLM to distill paraphrase-robust Diagnostic Semantic Tokens and to expand prompts into diagnosis-aware attribute bundles; and a Prototype stream that affords component-level morphological control via a prototype bank. On the data front, we curate a 2.65M image-text corpus and a finely annotated, high-quality 68K subset to alleviate data scarcity. For a comprehensive assessment, we establish a four-tier evaluation hierarchy tailored to pathology. Extensive experiments demonstrate UniPath's SOTA performance, including a Patho-FID of 80.9 (51% better than the second-best) and fine-grained semantic control achieving 98.7% of the real-image. The meticulously curated datasets, complete source code, and pre-trained model weights developed in this study will be made openly accessible to the public.

Figures (17)

• Figure 1: $\textsc{UniPath}$ achieves overall leading performance against state-of-the-art baselines across our four-tier evaluation hierarchy.
• Figure 2: Overview of $\textsc{UniPath}$: Unifying pathology understanding and synthesis via Multi-Stream Control. (a): Data governance and prototype retrieval. (b): A frozen pathology MLLM (Understanding) steers a (c): generative DiT (Generation) through MSC fusing raw-text, high-level semantics (emitting diagnostic semantic tokens via learnable queries), and prototype cues.
• Figure 3: MLLM and human expert evaluation results.
• Figure 4: Few-shot classification on Kather-CRC-2016.
• Figure 5: Dataset and component ablation results.