Attention-Map Augmentation for Hypercomplex Breast Cancer Classification

TL;DR

This work tackles the difficulty of whole-mammogram breast cancer classification by focusing the model on the small ROI using attention maps. It introduces the PHAM framework, which augments each image with an attention map and processes the multi-dimensional input through parameterized hypercomplex neural networks (PHNNs), enabling the model to learn both global and local relations between the image and its attention map. By employing PHResNets with weights defined in a hypercomplex algebra, the method reduces parameters by a factor of 1/$n$ and effectively models correlations across channels, improving performance on mammography and histopathology datasets while using far fewer parameters than real-valued baselines. The experimental results demonstrate state-of-the-art performance versus attention-based methods and show strong generalization across data modalities, underscoring the practical value of attention-map conditioning in medical image classification.

Abstract

Breast cancer is the most widespread neoplasm among women and early detection of this disease is critical. Deep learning techniques have become of great interest to improve diagnostic performance. However, distinguishing between malignant and benign masses in whole mammograms poses a challenge, as they appear nearly identical to an untrained eye, and the region of interest (ROI) constitutes only a small fraction of the entire image. In this paper, we propose a framework, parameterized hypercomplex attention maps (PHAM), to overcome these problems. Specifically, we deploy an augmentation step based on computing attention maps. Then, the attention maps are used to condition the classification step by constructing a multi-dimensional input comprised of the original breast cancer image and the corresponding attention map. In this step, a parameterized hypercomplex neural network (PHNN) is employed to perform breast cancer classification. The framework offers two main advantages. First, attention maps provide critical information regarding the ROI and allow the neural model to concentrate on it. Second, the hypercomplex architecture has the ability to model local relations between input dimensions thanks to hypercomplex algebra rules, thus properly exploiting the information provided by the attention map. We demonstrate the efficacy of the proposed framework on both mammography images as well as histopathological ones. We surpass attention-based state-of-the-art networks and the real-valued counterpart of our approach. The code of our work is available at https://github.com/ispamm/AttentionBCS.

Figures (5)

• Figure 1: Top rows: attention maps of INbreast obtained from PatchConvNet fine-tuned on CBIS-DDSM. Bottom rows: attention maps of CBIS-DDSM obtained from PatchConvNet fine-tuned on INbreast. The left column comprises mammograms with a malignant finding, while the right presents negative/benign mammograms.
• Figure 2: PHAM framework. On the left, the attention-map augmentation step is depicted. Herein, attention maps are computed offline with the fine-tuned PatchConvNet model. Then, they are used to perform a form of conditioning on the hypercomplex model. On the right, a PHResNet with $n=2$ for mammography images ($n=4$ for histopathology images) is employed as the backbone to perform breast cancer classification. It is able to model relations between breast cancer images and attention maps, as can be seen in the visualization of the PHC layer.
• Figure 3: ROC curve corresponding to the best run of models with AM for experiments conducted on INbreast and CBIS-DDSM datasets.
• Figure 4: Confusion matrices of models with AM for experiments on INbreast and CBIS-DDSM.
• Figure 5: Confusion matrices of models with AM for experiments on BreakHis.