Polyp-Gen: Realistic and Diverse Polyp Image Generation for Endoscopic Dataset Expansion

TL;DR

Polyp-Gen tackles data scarcity and privacy barriers in endoscopic polyp detection by delivering a fully automatic diffusion-based image generation framework. It introduces a boundary-enhanced training scheme with a lesion-guided loss and a hierarchical retrieval-based sampling strategy to model plausible polyp locations without clinical priors. Empirical results show state-of-the-art realism and diversity (low FID, high IS) and meaningful improvements in downstream polyp detection when synthetic data are added, with strong zero-shot generalization to external datasets. The approach holds practical potential for expanding ADS training data while preserving realism and diversity across endoscopic images.

Abstract

Automated diagnostic systems (ADS) have shown significant potential in the early detection of polyps during endoscopic examinations, thereby reducing the incidence of colorectal cancer. However, due to high annotation costs and strict privacy concerns, acquiring high-quality endoscopic images poses a considerable challenge in the development of ADS. Despite recent advancements in generating synthetic images for dataset expansion, existing endoscopic image generation algorithms failed to accurately generate the details of polyp boundary regions and typically required medical priors to specify plausible locations and shapes of polyps, which limited the realism and diversity of the generated images. To address these limitations, we present Polyp-Gen, the first full-automatic diffusion-based endoscopic image generation framework. Specifically, we devise a spatial-aware diffusion training scheme with a lesion-guided loss to enhance the structural context of polyp boundary regions. Moreover, to capture medical priors for the localization of potential polyp areas, we introduce a hierarchical retrieval-based sampling strategy to match similar fine-grained spatial features. In this way, our Polyp-Gen can generate realistic and diverse endoscopic images for building reliable ADS. Extensive experiments demonstrate the state-of-the-art generation quality, and the synthetic images can improve the downstream polyp detection task. Additionally, our Polyp-Gen has shown remarkable zero-shot generalizability on other datasets. The source code is available at https://github.com/CUHK-AIM-Group/Polyp-Gen.

Figures (6)

• Figure 1: Examples of the generated endoscopic images by our Polyp-Gen. Existing endoscopic image generation methods failed to accurately generate the details of polyp boundary regions, while our Polyp-Gen framework achieves realistic and diverse endoscopic image generations.
• Figure 2: Overview of Polyp-Gen. (1) We devise a spatial-aware diffusion training scheme to enhance the structural context of polyp boundary regions while preserving endoscopic global information. (2) We introduce a hierarchical retrieval-based sampling strategy to adaptively determine potential polyp locations for polyp generation.
• Figure 3: Construction of database. We utilize the trained Polyp-Gen to convert polyp images into normal images, thereby constructing a database for mask proposals.
• Figure 4: Showcases of Polyp-Gen. The generated images exhibit remarkable realism and diversity.
• Figure 5: Qualitative comparison of polyp image generation on LDPolypVideo and Kvasir-Seg datasets.