Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deeply Supervised Skin Lesions Diagnosis with Stage and Branch Attention

Wei Dai, Rui Liu, Tianyi Wu, Min Wang, Jianqin Yin, Jun Liu

TL;DR

This work addresses the need for accurate yet portable skin-lesion classification by introducing HierAttn, a lightweight, deeply supervised architecture that combines stage attention, branch attention, and same-channel attention (SCAttn) with a convolution–transformer hybrid. It achieves top performance among mobile-friendly models on ISIC2019 and PAD2020, delivering 96.70% and 91.22% accuracy (with AUCs up to 0.9972 and 0.9882, respectively) while maintaining a small parameter count (~1.08M for HierAttn_s). The approach relies on a single training loss and leverages hierarchical pooling to learn local and global representations without increasing model size or training complexity. The results demonstrate strong potential for point-of-care and mobile dermoscopy, enabling rapid, accessible skin-lesion diagnostics with high reliability.

Abstract

Accurate and unbiased examinations of skin lesions are critical for the early diagnosis and treatment of skin diseases. Visual features of skin lesions vary significantly because the images are collected from patients with different lesion colours and morphologies by using dissimilar imaging equipment. Recent studies have reported that ensembled convolutional neural networks (CNNs) are practical to classify the images for early diagnosis of skin disorders. However, the practical use of these ensembled CNNs is limited as these networks are heavyweight and inadequate for processing contextual information. Although lightweight networks (e.g., MobileNetV3 and EfficientNet) were developed to achieve parameters reduction for implementing deep neural networks on mobile devices, insufficient depth of feature representation restricts the performance. To address the existing limitations, we develop a new lite and effective neural network, namely HierAttn. The HierAttn applies a novel deep supervision strategy to learn the local and global features by using multi-stage and multi-branch attention mechanisms with only one training loss. The efficacy of HierAttn was evaluated by using the dermoscopy images dataset ISIC2019 and smartphone photos dataset PAD-UFES-20 (PAD2020). The experimental results show that HierAttn achieves the best accuracy and area under the curve (AUC) among the state-of-the-art lightweight networks. The code is available at https://github.com/anthonyweidai/HierAttn.

Deeply Supervised Skin Lesions Diagnosis with Stage and Branch Attention

TL;DR

This work addresses the need for accurate yet portable skin-lesion classification by introducing HierAttn, a lightweight, deeply supervised architecture that combines stage attention, branch attention, and same-channel attention (SCAttn) with a convolution–transformer hybrid. It achieves top performance among mobile-friendly models on ISIC2019 and PAD2020, delivering 96.70% and 91.22% accuracy (with AUCs up to 0.9972 and 0.9882, respectively) while maintaining a small parameter count (~1.08M for HierAttn_s). The approach relies on a single training loss and leverages hierarchical pooling to learn local and global representations without increasing model size or training complexity. The results demonstrate strong potential for point-of-care and mobile dermoscopy, enabling rapid, accessible skin-lesion diagnostics with high reliability.

Abstract

Accurate and unbiased examinations of skin lesions are critical for the early diagnosis and treatment of skin diseases. Visual features of skin lesions vary significantly because the images are collected from patients with different lesion colours and morphologies by using dissimilar imaging equipment. Recent studies have reported that ensembled convolutional neural networks (CNNs) are practical to classify the images for early diagnosis of skin disorders. However, the practical use of these ensembled CNNs is limited as these networks are heavyweight and inadequate for processing contextual information. Although lightweight networks (e.g., MobileNetV3 and EfficientNet) were developed to achieve parameters reduction for implementing deep neural networks on mobile devices, insufficient depth of feature representation restricts the performance. To address the existing limitations, we develop a new lite and effective neural network, namely HierAttn. The HierAttn applies a novel deep supervision strategy to learn the local and global features by using multi-stage and multi-branch attention mechanisms with only one training loss. The efficacy of HierAttn was evaluated by using the dermoscopy images dataset ISIC2019 and smartphone photos dataset PAD-UFES-20 (PAD2020). The experimental results show that HierAttn achieves the best accuracy and area under the curve (AUC) among the state-of-the-art lightweight networks. The code is available at https://github.com/anthonyweidai/HierAttn.
Paper Structure (32 sections, 2 equations, 9 figures, 4 tables)

This paper contains 32 sections, 2 equations, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. CNNs in skin lesions diagnosis
  4. Deeply supervised learning
  5. Attention mechanisms
  6. Vision transformer
  7. Lightweight networks
  8. Proposed Methodology
  9. Skin lesion dataset
  10. Pre-processing
  11. Image pre-processing
  12. Data balance
  13. HierAttn architecture
  14. Branch attention
  15. Same channel attention
  16. ...and 17 more sections

Figures (9)

  • Figure 1: HierAttn architecture. Conv-$n\times n$ represents a standard convolution, and SCADW refers to a depthwise separable convolution block with the SCAttn module. Down-sampling blocks are marked with ↓2. Stage attention has a SCADW block followed by a CTH block (pink block) that thoroughly aggregates local features and learns contextual representations. Branch attention utilises hierarchical pooling to extract global and local features steadily.
  • Figure 2: Data distribution on ISIC2019 (left image) and PAD2020 (right image)
  • Figure 3: The progress in cropping image (a) original image, (b) greyed image, (c) binarised image, and (d) cropped image.
  • Figure 4: Branch attention by hierarchical pooling, assembling and randomising tensors from different branches.
  • Figure 5: Schematic comparison of the original block (without attention mechanism) (a), the SEAttn block (b), and the proposed SCAttn block (c).
  • ...and 4 more figures