Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Label-efficient Contrastive Learning-based model for nuclei detection and classification in 3D Cardiovascular Immunofluorescent Images

Nazanin Moradinasab, Rebecca A. Deaton, Laura S. Shankman, Gary K. Owens, Donald E. Brown

TL;DR

The paper tackles automated detection and classification of nuclei in 3D multichannel immunofluorescent images under a weakly supervised setting. It introduces LECL, a label-efficient framework that combines Extended Maximum Intensity Projection (EMIP) to generate nucleus-aware 2D representations from 3D data and Supervised Contrastive Learning (SCL) to learn discriminative features, built on the Hover-Net backbone. The training objective blends cross-entropy, Dice, SupCon, and entropy losses as $L = L^{CE}+L^{Dice}+L^{SupCon}+L^{entropy}$ with an explicit $L^{SCL}$ term guiding pixel embeddings, while EMIP uses Voronoi-based boundaries and 3D distance maps to isolate each nucleus in projection. Experiments on D1 and D2 show LECL outperforms MIP-based approaches and prior weakly supervised methods in detection and classification, and ablations confirm the contributions of EMIP and SCL in reducing false overlaps and improving inter-class separability. The work enables more accurate, cost-effective 3D nuclei analysis in cardiovascular pathology and suggests directions for synthetic data, domain adaptation, and application to additional markers.

Abstract

Recently, deep learning-based methods achieved promising performance in nuclei detection and classification applications. However, training deep learning-based methods requires a large amount of pixel-wise annotated data, which is time-consuming and labor-intensive, especially in 3D images. An alternative approach is to adapt weak-annotation methods, such as labeling each nucleus with a point, but this method does not extend from 2D histopathology images (for which it was originally developed) to 3D immunofluorescent images. The reason is that 3D images contain multiple channels (z-axis) for nuclei and different markers separately, which makes training using point annotations difficult. To address this challenge, we propose the Label-efficient Contrastive learning-based (LECL) model to detect and classify various types of nuclei in 3D immunofluorescent images. Previous methods use Maximum Intensity Projection (MIP) to convert immunofluorescent images with multiple slices to 2D images, which can cause signals from different z-stacks to falsely appear associated with each other. To overcome this, we devised an Extended Maximum Intensity Projection (EMIP) approach that addresses issues using MIP. Furthermore, we performed a Supervised Contrastive Learning (SCL) approach for weakly supervised settings. We conducted experiments on cardiovascular datasets and found that our proposed framework is effective and efficient in detecting and classifying various types of nuclei in 3D immunofluorescent images.

Label-efficient Contrastive Learning-based model for nuclei detection and classification in 3D Cardiovascular Immunofluorescent Images

TL;DR

The paper tackles automated detection and classification of nuclei in 3D multichannel immunofluorescent images under a weakly supervised setting. It introduces LECL, a label-efficient framework that combines Extended Maximum Intensity Projection (EMIP) to generate nucleus-aware 2D representations from 3D data and Supervised Contrastive Learning (SCL) to learn discriminative features, built on the Hover-Net backbone. The training objective blends cross-entropy, Dice, SupCon, and entropy losses as with an explicit term guiding pixel embeddings, while EMIP uses Voronoi-based boundaries and 3D distance maps to isolate each nucleus in projection. Experiments on D1 and D2 show LECL outperforms MIP-based approaches and prior weakly supervised methods in detection and classification, and ablations confirm the contributions of EMIP and SCL in reducing false overlaps and improving inter-class separability. The work enables more accurate, cost-effective 3D nuclei analysis in cardiovascular pathology and suggests directions for synthetic data, domain adaptation, and application to additional markers.

Abstract

Recently, deep learning-based methods achieved promising performance in nuclei detection and classification applications. However, training deep learning-based methods requires a large amount of pixel-wise annotated data, which is time-consuming and labor-intensive, especially in 3D images. An alternative approach is to adapt weak-annotation methods, such as labeling each nucleus with a point, but this method does not extend from 2D histopathology images (for which it was originally developed) to 3D immunofluorescent images. The reason is that 3D images contain multiple channels (z-axis) for nuclei and different markers separately, which makes training using point annotations difficult. To address this challenge, we propose the Label-efficient Contrastive learning-based (LECL) model to detect and classify various types of nuclei in 3D immunofluorescent images. Previous methods use Maximum Intensity Projection (MIP) to convert immunofluorescent images with multiple slices to 2D images, which can cause signals from different z-stacks to falsely appear associated with each other. To overcome this, we devised an Extended Maximum Intensity Projection (EMIP) approach that addresses issues using MIP. Furthermore, we performed a Supervised Contrastive Learning (SCL) approach for weakly supervised settings. We conducted experiments on cardiovascular datasets and found that our proposed framework is effective and efficient in detecting and classifying various types of nuclei in 3D immunofluorescent images.
Paper Structure (9 sections, 1 equation, 5 figures, 4 tables)

This paper contains 9 sections, 1 equation, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Challenges
  3. Method
  4. Extended Maximum Intensity Projection (EMIP)
  5. Supervised Contrastive Learning-based (SCL) training strategy
  6. Experimental Results
  7. Discussion
  8. Acknowledgments
  9. Appendix

Figures (5)

  • Figure 1: A) Challenges: (1) nucleus in the yellow square: Even though the ground truth labels for the nucleus in the yellow and green squares are positive, they are only coincident in the second slice, (2) nucleus in the white square: it shows an example of nonoverlapping marker and nucleus, (B-1) The MIP approach's output, (B-2) The EMIP approach's output, (B-3) Ground truth point annotations: green color represents nuclei labeled as positive, and white color represents nuclei labeled as negative.
  • Figure 2: Schematic representation of the LECL model
  • Figure 3: (A-1) The nuclei z-slices, (A-2) The marker z-slices, (B-1) The Voronoi label, (B-2) The Voronoi Cell (VC) binary mask associated to convex cell j that assigns label 1 to convex cell j and zero to others, (B-3) The z-slice 6 of nuclei channel, (B-4) The multiplication's output of VC mask and z-slice 6 which depicts the nucleus located in convex cell j, (C-1) nuclei z-slices, (C-2) The 3D binary mask, (D-1) The intersection between VC mask (B-2) and 3D binary mask (C-2), (D-2) the intersection between the VC binary mask and the nuclei/marker z-slices, (D-3) EMIP output
  • Figure 4: a) It presents the sequence of channels (i.e. z=0,..,n) for the nuclei (first row) and the Lineage Tracing marker (second row). The nuclei and Lineage Tracing marker channels are associated with each other in order. The third row indicates the linear combination of the nuclei and Lineage Tracing marker per slice.
  • Figure 5: The model's output trained using the HoVer-Net(MIP) and HoVer-Net(EMIP). The HoVer-Net (EMIP) model correctly predicts label positive for nuclei in the yellow circle, which is consistent with the ground truth labels. In contrast, HoVer-Net (MIP) incorrectly predicts these nuclei as negative. Both models incorrectly predict the nuclei's labels in the blue circle.