Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NCT-CRC-HE: Not All Histopathological Datasets Are Equally Useful

Andrey Ignatov, Grigory Malivenko

TL;DR

The paper scrutinizes the NCT-CRC-HE-100K dataset to determine whether high DL performance in histopathology reflects tissue morphology or dataset artifacts. It identifies strong color signatures after stain normalization, varying JPEG artifacts, and corrupted patches from dynamic-range handling that can drive model decisions. A set of lightweight baselines using mean RGB values, color histograms, and ImageNet-derived features shows that color and generic texture cues capture substantial task information, with EfficientNet-B0 achieving 97.7% (single) to 98.3% (ensemble) accuracy, suggesting limited benefit from morphology-specific modeling on this dataset. The authors advocate cautious interpretation of results on this dataset, emphasize bias-aware evaluation and dataset quality improvements, and provide code and data to enable replication and further scrutiny.

Abstract

Numerous deep learning-based solutions have been proposed for histopathological image analysis over the past years. While they usually demonstrate exceptionally high accuracy, one key question is whether their precision might be affected by low-level image properties not related to histopathology but caused by microscopy image handling and pre-processing. In this paper, we analyze a popular NCT-CRC-HE-100K colorectal cancer dataset used in numerous prior works and show that both this dataset and the obtained results may be affected by data-specific biases. The most prominent revealed dataset issues are inappropriate color normalization, severe JPEG artifacts inconsistent between different classes, and completely corrupted tissue samples resulting from incorrect image dynamic range handling. We show that even the simplest model using only 3 features per image (red, green and blue color intensities) can demonstrate over 50% accuracy on this 9-class dataset, while using color histogram not explicitly capturing cell morphology features yields over 82% accuracy. Moreover, we show that a basic EfficientNet-B0 ImageNet pretrained model can achieve over 97.7% accuracy on this dataset, outperforming all previously proposed solutions developed for this task, including dedicated foundation histopathological models and large cell morphology-aware neural networks. The NCT-CRC-HE dataset is publicly available and can be freely used to replicate the presented results. The codes and pre-trained models used in this paper are available at https://github.com/gmalivenko/NCT-CRC-HE-experiments

NCT-CRC-HE: Not All Histopathological Datasets Are Equally Useful

TL;DR

The paper scrutinizes the NCT-CRC-HE-100K dataset to determine whether high DL performance in histopathology reflects tissue morphology or dataset artifacts. It identifies strong color signatures after stain normalization, varying JPEG artifacts, and corrupted patches from dynamic-range handling that can drive model decisions. A set of lightweight baselines using mean RGB values, color histograms, and ImageNet-derived features shows that color and generic texture cues capture substantial task information, with EfficientNet-B0 achieving 97.7% (single) to 98.3% (ensemble) accuracy, suggesting limited benefit from morphology-specific modeling on this dataset. The authors advocate cautious interpretation of results on this dataset, emphasize bias-aware evaluation and dataset quality improvements, and provide code and data to enable replication and further scrutiny.

Abstract

Numerous deep learning-based solutions have been proposed for histopathological image analysis over the past years. While they usually demonstrate exceptionally high accuracy, one key question is whether their precision might be affected by low-level image properties not related to histopathology but caused by microscopy image handling and pre-processing. In this paper, we analyze a popular NCT-CRC-HE-100K colorectal cancer dataset used in numerous prior works and show that both this dataset and the obtained results may be affected by data-specific biases. The most prominent revealed dataset issues are inappropriate color normalization, severe JPEG artifacts inconsistent between different classes, and completely corrupted tissue samples resulting from incorrect image dynamic range handling. We show that even the simplest model using only 3 features per image (red, green and blue color intensities) can demonstrate over 50% accuracy on this 9-class dataset, while using color histogram not explicitly capturing cell morphology features yields over 82% accuracy. Moreover, we show that a basic EfficientNet-B0 ImageNet pretrained model can achieve over 97.7% accuracy on this dataset, outperforming all previously proposed solutions developed for this task, including dedicated foundation histopathological models and large cell morphology-aware neural networks. The NCT-CRC-HE dataset is publicly available and can be freely used to replicate the presented results. The codes and pre-trained models used in this paper are available at https://github.com/gmalivenko/NCT-CRC-HE-experiments
Paper Structure (14 sections, 7 figures, 3 tables)

This paper contains 14 sections, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Exploring and Analyzing the NCT-CRC-HE Dataset
  3. RGB Channel Intensities and Color Distribution
  4. JPEG Compression Artifacts
  5. Corrupted Images
  6. Other Potential Issues
  7. Proposed Method
  8. Experimental Results
  9. Baseline Solution 1: Using R, G and B Color Intensities
  10. Baseline Solution 2: Using Color Histograms
  11. Baseline Solution 3: Using ImageNet Features
  12. EfficientNet-B0 Based Solution
  13. Estimating the Effect of JPEG Compression Artifacts and Color Bias on Model Predictions
  14. Conclusion

Figures (7)

  • Figure 1: Visualization of normalized H&E stained image patches from the NCT-CRC-HE-100K dataset. The images were sampled randomly for each of 9 tissue classes.
  • Figure 2: Visualized average red, green and blue color intensities for NCT-CRC-HE training images. Top row shows 2D projections to the corresponding color spaces.
  • Figure 3: Visualized average red, green and blue color intensities for NCT-CRC-HE test images. Top row shows 2D projections to the corresponding color spaces.
  • Figure 4: Visualized color histograms for each NCT-CRC-HE tissue class, training set.
  • Figure 5: Visualized color histograms for each NCT-CRC-HE tissue class, validation set.
  • ...and 2 more figures