Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Level Bidirectional Decoder Interaction for Uncertainty-Aware Breast Ultrasound Analysis

Abdullah Al Shafi, Md Kawsar Mahmud Khan Zunayed, Safin Ahmmed, Sk Imran Hossain, Engelbert Mephu Nguifo

TL;DR

A multi-task framework addressing limitations through multi-level decoder interaction and uncertainty-aware adaptive coordination is proposed, validating that decoder-level bidirectional communication is more effective than conventional encoder-only parameter sharing.

Abstract

Breast ultrasound interpretation requires simultaneous lesion segmentation and tissue classification. However, conventional multi-task learning approaches suffer from task interference and rigid coordination strategies that fail to adapt to instance-specific prediction difficulty. We propose a multi-task framework addressing these limitations through multi-level decoder interaction and uncertainty-aware adaptive coordination. Task Interaction Modules operate at all decoder levels, establishing bidirectional segmentation-classification communication during spatial reconstruction through attention weighted pooling and multiplicative modulation. Unlike prior single-level or encoder-only approaches, this multi-level design captures scale specific task synergies across semantic-to-spatial scales, producing complementary task interaction streams. Uncertainty-Proxy Attention adaptively weights base versus enhanced features at each level using feature activation variance, enabling per-level and per-sample task balancing without heuristic tuning. To support instance-adaptive prediction, multi-scale context fusion captures morphological cues across varying lesion sizes. Evaluation on multiple publicly available breast ultrasound datasets demonstrates competitive performance, including 74.5% lesion IoU and 90.6% classification accuracy on BUSI dataset. Ablation studies confirm that multi-level task interaction provides significant performance gains, validating that decoder-level bidirectional communication is more effective than conventional encoder-only parameter sharing. The code is available at: https://github.com/C-loud-Nine/Uncertainty-Aware-Multi-Level-Decoder-Interaction.

Multi-Level Bidirectional Decoder Interaction for Uncertainty-Aware Breast Ultrasound Analysis

TL;DR

A multi-task framework addressing limitations through multi-level decoder interaction and uncertainty-aware adaptive coordination is proposed, validating that decoder-level bidirectional communication is more effective than conventional encoder-only parameter sharing.

Abstract

Breast ultrasound interpretation requires simultaneous lesion segmentation and tissue classification. However, conventional multi-task learning approaches suffer from task interference and rigid coordination strategies that fail to adapt to instance-specific prediction difficulty. We propose a multi-task framework addressing these limitations through multi-level decoder interaction and uncertainty-aware adaptive coordination. Task Interaction Modules operate at all decoder levels, establishing bidirectional segmentation-classification communication during spatial reconstruction through attention weighted pooling and multiplicative modulation. Unlike prior single-level or encoder-only approaches, this multi-level design captures scale specific task synergies across semantic-to-spatial scales, producing complementary task interaction streams. Uncertainty-Proxy Attention adaptively weights base versus enhanced features at each level using feature activation variance, enabling per-level and per-sample task balancing without heuristic tuning. To support instance-adaptive prediction, multi-scale context fusion captures morphological cues across varying lesion sizes. Evaluation on multiple publicly available breast ultrasound datasets demonstrates competitive performance, including 74.5% lesion IoU and 90.6% classification accuracy on BUSI dataset. Ablation studies confirm that multi-level task interaction provides significant performance gains, validating that decoder-level bidirectional communication is more effective than conventional encoder-only parameter sharing. The code is available at: https://github.com/C-loud-Nine/Uncertainty-Aware-Multi-Level-Decoder-Interaction.
Paper Structure (12 sections, 7 equations, 3 figures, 2 tables)

This paper contains 12 sections, 7 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Task Interaction Module
  4. Uncertainty Proxy Attention
  5. Multi-Scale Context and Attention Mechanisms
  6. Multi-Task Loss Formulation
  7. Experiments and Results
  8. Datasets and Implementation
  9. Quantitative Evaluation
  10. Ablation Studies
  11. Discussion and Conclusion

Figures (3)

  • Figure 1: Overview of the proposed architecture. Multi-scale fusion modules augment encoder features. TIM enables bidirectional task interaction at all four decoder levels ($D_1$--$D_4$). UPA adaptively modulates information flow at each level. Dual heads generate segmentation masks and classification predictions jointly.
  • Figure 2: Qualitative segmentation results on BUSI al2020dataset. Columns show the input image, ground truth, and predictions from U-Net ronneberger2015unet, Att-U-Net oktay2018attention, UNet++ zhou2018unetpp, Swin-UNet cao2022swin, and the proposed method.
  • Figure 3: Analysis of TIM and UPA modules across decoder levels. (Left) $\ell_2$ displacement indicates dominant segmentation-to-classification flow. (Right) UPA weight distributions show adaptive task balancing: higher levels assign greater weights to segmentation.