DeNuC: Decoupling Nuclei Detection and Classification in Histopathology

TL;DR

DeNuC is proposed, a simple yet effective method designed to break through existing bottlenecks by Decoupling Nuclei detection and Classification that effectively unlocks the representational potential of FMs for NDC and significantly outperforms state-of-the-art methods.

Abstract

Pathology Foundation Models (FMs) have shown strong performance across a wide range of pathology image representation and diagnostic tasks. However, FMs do not exhibit the expected performance advantage over traditional specialized models in Nuclei Detection and Classification (NDC). In this work, we reveal that jointly optimizing nuclei detection and classification leads to severe representation degradation in FMs. Moreover, we identify that the substantial intrinsic disparity in task difficulty between nuclei detection and nuclei classification renders joint NDC optimization unnecessarily computationally burdensome for the detection stage. To address these challenges, we propose DeNuC, a simple yet effective method designed to break through existing bottlenecks by Decoupling Nuclei detection and Classification. DeNuC employs a lightweight model for accurate nuclei localization, subsequently leveraging a pathology FM to encode input images and query nucleus-specific features based on the detected coordinates for classification. Extensive experiments on three widely used benchmarks demonstrate that DeNuC effectively unlocks the representational potential of FMs for NDC and significantly outperforms state-of-the-art methods. Notably, DeNuC improves F1 scores by 4.2% and 3.6% (or higher) on the BRCAM2C and PUMA datasets, respectively, while using only 16% (or fewer) trainable parameters compared to other methods. Code is available at https://github.com/ZijiangY1116/DeNuC.

Figures (3)

• Figure 1: Analysis of model performance during the NDC training. (a) We evaluate the representation capability of UNI2-H uni during joint optimization for NDC on the OCELOT ryu2023ocelot dataset via linear probing. UNI2-H undergoes a severe representation degradation in the early training phase. Although the performance subsequently recovers, it fails to regain its initial optimal level and rapidly deteriorates into over-fitting. (b) We evaluate the detection and classification F1 scores of a ConvNeXt-S convnext model pre-trained solely on ImageNet-1K throughout the training process on PUMA puma. Despite lacking domain-specific pathology pre-training, nuclei detection converges approximately 2.3 $\times$ faster than classification, highlighting the inherent difficulty disparity between the two tasks.
• Figure 2: Comparison of DeNuC and SOTA methods. DeNuC achieves significantly superior performance across three benchmark datasets.
• Figure 3: Illustration of (a) Joint optimization and (b) Decoupled optimization for NDC. Existing methods require the backbone to accommodate additional optimization objectives beyond nuclei representation, leading to representation degradation. In contrast, DeNuC employs an independent lightweight detection network $\mathcal{D}$ for nuclei localization, thereby allowing the backbone to focus exclusively on representation learning.