Entropy Bootstrapping for Weakly Supervised Nuclei Detection

TL;DR

This approach uses individual point labels for an entropy estimation to approximate an underlying distribution of cell pixels, and infer full cell masks from this distribution, and uses Mask-RCNN to produce an instance segmentation output.

Abstract

Microscopy structure segmentation, such as detecting cells or nuclei, generally requires a human to draw a ground truth contour around each instance. Weakly supervised approaches (e.g. consisting of only single point labels) have the potential to reduce this workload significantly. Our approach uses individual point labels for an entropy estimation to approximate an underlying distribution of cell pixels. We infer full cell masks from this distribution, and use Mask-RCNN to produce an instance segmentation output. We compare this point--annotated approach with training on the full ground truth masks. We show that our method achieves a comparatively good level of performance, despite a 95% reduction in pixel labels.

Figures (8)

• Figure 1: Mask-RCNN outputs. Left column: three separate images with ground truth masks overlaid. Middle column: predictions from training Mask-RCNN on full masks directly. Right column: predictions from training on our watershed instance masks as described in \ref{['sec:maskrcnnassess']}. The bottom image was chosen to show a universal failure case where both models detect false positives despite no nuclei being present.
• Figure 2: Figure showing the overall pipeline. The entropy estimation method we are using Carvalho_2020_CVPR is trained on the point annotations shown in orange within a) to generate entropy shown in b). This is converted deterministically into instance masks as in c). Finally we use Mask-RCNN trained on these instance masks to detect nuclei as shown in d).
• Figure 3: Figure showing the instance mask generation from the entropy shown in a). We use adaptive thresholding and the Voronoi edges to convert our distribution a) into a binary segmentation and we find the regions of interest (ROIs), the locations of the discrete connected elements of the segmentation. These are shown as green boxes in b). We then remove small artifacts and watershed to get instance masks as in c). These instance masks are then checked against the point annotations to remove erroneous instances as shown in d). Finally we output all the instances which are co-located with the point annotations, shown in e).
• Figure 4: Figure illustrating assessment of entropy as an approximation of nucleus pixel distribution. We show here a test image, its corresponding point annotations, the test entropy and the ground truth. We also show how the Dice score varies with threshold value and an ROC plot.
• Figure 5: Figure illustrating assessment of instance masks generated from entropy. We show a test image with the ground truth instance masks outlined in orange, the test entropy overlaid with the Voronoi edges, the binary mask from adaptive thresholding and the output of the watershed instancing. We then show a visualisation comparing the ground truth bounding boxes in orange with the watershed bounding boxes in grey. The intersection of the two is shown in green. Finally we show a plot of nuclei detection rate against IoU threshold.