Case-level Breast Cancer Prediction for Real Hospital Settings

TL;DR

A framework for case-level breast cancer prediction that does not require any manual annotation and can be trained with case labels readily available at the hospital, and can be applied in real hospital settings where only case labels, and a variable number of images per case are available, without any loss in performance compared to models trained on image labels.

Abstract

Breast cancer prediction models for mammography assume that annotations are available for individual images or regions of interest (ROIs), and that there is a fixed number of images per patient. These assumptions do not hold in real hospital settings, where clinicians provide only a final diagnosis for the entire mammography exam (case). Since data in real hospital settings scales with continuous patient intake, while manual annotation efforts do not, we develop a framework for case-level breast cancer prediction that does not require any manual annotation and can be trained with case labels readily available at the hospital. Specifically, we propose a two-level multi-instance learning (MIL) approach at patch and image level for case-level breast cancer prediction and evaluate it on two public and one private dataset. We propose a novel domain-specific MIL pooling observing that breast cancer may or may not occur in both sides, while images of both breasts are taken as a precaution during mammography. We propose a dynamic training procedure for training our MIL framework on a variable number of images per case. We show that our two-level MIL model can be applied in real hospital settings where only case labels, and a variable number of images per case are available, without any loss in performance compared to models trained on image labels. Only trained with weak (case-level) labels, it has the capability to point out in which breast side, mammography view and view region the abnormality lies.

Figures (15)

• Figure 1: Existing breast cancer prediction models differ at the input levels and are trained with labels at different granularities: ROI, image, side and case level (cf. Table \ref{['tab:relatedwork']}). Here, we show how real hospital settings are different from the settings of existing work. In real hospital settings, labels are available at the case level and cases can contain a variable number of images (last row) and we propose a model for this setting.
• Figure 2: A malignant case from the CLaM dataset showing standard craniocaudal (CC) and mediolateral oblique (MLO) views of the right (R-) and left (L-) breast, along with an additional exaggerated craniocaudal view of the right breast (R-XCCL). A pathologically proven malignant mass of irregular shape and indistinct margin with calcification is visible mainly in the R-MLO and R-XCCL views and no abnormality is visible in the left breast. The malignant mass is best visible in the additional view, R-XCCL, highlighting the importance of including additional views in the input to a predictive model. The case is labeled malignant in the hospital system due to the presence of malignant abnormalities in the right breast. This highlights the challenges C1 (not all images contain abnormalities), C2 (ROIs are small) and C3 (variable number of images per case).
• Figure 3: Model architecture. Center: overall architecture. An end-to-end trained feature extractor is applied to a variable number of input images (e.g. L-CC, L-MLO, R-CC), returning a feature representation per image. Image-level multi-instance learning (MIL) methods then aggregate these feature for a final decision. Left: An example feature extractor module, GMIC, capable of unsupervised ROI extraction. Right: Our proposed domain-specific pooling block performing view-level pooling and then performing side-level pooling (based on the assumption that single images of one side highly correlate in the presence of abnormality). Dashed lines indicate absent views. Best viewed in color.
• Figure 4: Entropy of attention score distribution of attention-based models for malignant and benign class for CLaM-FV. Red dashed line shows the entropy for uniform weights of all views, i.e., attention score of 0.25 for four views. Bars for the malignant class are away from uniform distribution indicating that the models can differentiate well among the images in malignant cases.
• Figure 5: Agreement of images identified as relevant, i.e., malignant (attention score $>0.25$ or image probability $>0.5$ for IS models), for truly malignant cases. Our proposed model ES-Att$^{\text{side}}$ generally outperforms other models in finding the malignant images.