Joint Stream: Malignant Region Learning for Breast Cancer Diagnosis

TL;DR

This paper proposes to classify the six essential indicating factors (ER, PR, HER2, ALN, HG, HG, MS) for early BC diagnosis using H\&E stained WSI's and proposes a malignant region learning attention network.

Abstract

Early diagnosis of breast cancer (BC) significantly contributes to reducing the mortality rate worldwide. The detection of different factors and biomarkers such as Estrogen receptor (ER), Progesterone receptor (PR), Human epidermal growth factor receptor 2 (HER2) gene, Histological grade (HG), Auxiliary lymph node (ALN) status, and Molecular subtype (MS) can play a significant role in improved BC diagnosis. However, the existing methods predict only a single factor which makes them less suitable to use in diagnosis and designing a strategy for treatment. In this paper, we propose to classify the six essential indicating factors (ER, PR, HER2, ALN, HG, MS) for early BC diagnosis using H\&E stained WSI's. To precisely capture local neighboring relationships, we use spatial and frequency domain information from the large patch size of WSI's malignant regions. Furthermore, to cater the variable number of regions of interest sizes and give due attention to each region, we propose a malignant region learning attention network. Our experimental results demonstrate that combining spatial and frequency information using the malignant region learning module significantly improves multi-factor and single-factor classification performance on publicly available datasets.

Figures (6)

• Figure 1: The pipeline of the proposed joint stream: malignant region learning for multi-label classification. After extracting patches of size ($x \times y$) from the malignant regions, we extract the spatial and frequency domain features using the TresNet-xl ridnik2021asymmetric. For the frequency domain features, we convert the patch's color space from RGB (Red, Green, Blue) to YCbCr (Luminance, Chroma: Blue, Chroma: Red), compute Discrete Fourier transform (DFT) and Discrete wavelet transform (DWT). Finally, we combine the spatial and frequency feature bags using the malignant region learning (MRL) module to obtain the multi-label prediction. Where A represents the output of the attention mechanism.
• Figure 2: Examples of different WSIs and their malignant ROIs with class labels from Breast Cancer Core Needle Biopsy (BNCB) dataset: The shapes, sizes, and numbers of malignant ROIs from all the WSIs are different. The class of Estrogen receptor (ER), Progesterone receptor (PR), Human epidermal growth factor receptor 2 (HER2) gene, Histological grade (HG), Auxiliary lymph node (ALN) status, and Molecular subtype (MS) are also given.
• Figure 3: The conversion of the spatial to frequency domain (DFT). a) original image in the spatial domain, b) single channel grayscale fourier frequency spectrum c) the real value of three channels of DFT, and d) thresholded image where the mean value of (c) is used as a threshold. The figures show that images (c) and (d) hold very specific nuclear density information.
• Figure 4: Neural Network Architecture of MIL, Gated-MIL, and MRL attention mechanism. The output range of the (a) MIL and (b) Gated-MIL is from (-1 to 1) but the range of (c) MRL mechanism exists between (0 to 1).
• Figure 5: Graphical representation comparing the results of the proposed attention mechanism against state-of-the-art methods.