Cracking the PUMA Challenge in 24 Hours with CellViT++ and nnU-Net

TL;DR

This work addresses panoptic segmentation in melanoma histopathology under a strict 24-hour development constraint. It introduces a two-model pipeline combining CellViT++ for nuclei detection and nnU-Net for tissue segmentation, augmented by pre-training nnU-Net on NSCLC to boost tissue segmentation performance. The key findings show a Dice score of 0.750 for tissue segmentation, while nuclei detection remains on par with baselines; pretraining significantly improves tissue segmentation, including necrosis detection that the baseline misses. The approach demonstrates that deployable, resource-efficient histopathology pipelines can achieve robust performance using out-of-the-box frameworks and external data, enabling rapid deployment in clinical research settings.

Abstract

Automatic tissue segmentation and nuclei detection is an important task in pathology, aiding in biomarker extraction and discovery. The panoptic segmentation of nuclei and tissue in advanced melanoma (PUMA) challenge aims to improve tissue segmentation and nuclei detection in melanoma histopathology. Unlike many challenge submissions focusing on extensive model tuning, our approach emphasizes delivering a deployable solution within a 24-hour development timeframe, using out-of-the-box frameworks. The pipeline combines two models, namely CellViT++ for nuclei detection and nnU-Net for tissue segmentation. Our results demonstrate a significant improvement in tissue segmentation, achieving a Dice score of 0.750, surpassing the baseline score of 0.629. For nuclei detection, we obtained results comparable to the baseline in both challenge tracks. The code is publicly available at https://github.com/TIO-IKIM/PUMA.

Figures (1)

• Figure 1: Pipeline overview consisting of $\text{CellViT}^{{ ++}}$ for nuclei detection and nnU-Net with 2-stage training for tissue segmentation. Created in BioRender biorender_figure.