Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Attention-guided Adaptive Subsampling

Sharath M Shankaranarayana, Soumava Kumar Roy, Prasad Sudhakar, Chandan Aladahalli

TL;DR

This paper addresses the computational burden of processing high-dimensional medical data by introducing Deep Attention-guided Subsampling (DAS), a framework that performs input-adaptive, differentiable subsampling of frames or slices. DAS integrates a lightweight feature extractor, a multi-head attention layer, and a differentiable Gumbel-Softmax sampling mechanism with adaptive temperature to select a subset of informative frames during inference. Through extensive experiments on eight datasets, DAS generally outperforms prior learnable subsampling methods and can approach full-sequence performance while reducing computation, with notable robustness in real-world noisy clinical data. The approach also yields an interpretable sampling matrix, and future work points to learnable feature extraction to further enhance saliency-guided sampling across broader tasks.

Abstract

Although deep neural networks have provided impressive gains in performance, these improvements often come at the cost of increased computational complexity and expense. In many cases, such as 3D volume or video classification tasks, not all slices or frames are necessary due to inherent redundancies. To address this issue, we propose a novel learnable subsampling framework that can be integrated into any neural network architecture. Subsampling, being a nondifferentiable operation, poses significant challenges for direct adaptation into deep learning models. While some works, have proposed solutions using the Gumbel-max trick to overcome the problem of non-differentiability, they fall short in a crucial aspect: they are only task-adaptive and not inputadaptive. Once the sampling mechanism is learned, it remains static and does not adjust to different inputs, making it unsuitable for real-world applications. To this end, we propose an attention-guided sampling module that adapts to inputs even during inference. This dynamic adaptation results in performance gains and reduces complexity in deep neural network models. We demonstrate the effectiveness of our method on 3D medical imaging datasets from MedMNIST3D as well as two ultrasound video datasets for classification tasks, one of them being a challenging in-house dataset collected under real-world clinical conditions.

Deep Attention-guided Adaptive Subsampling

TL;DR

This paper addresses the computational burden of processing high-dimensional medical data by introducing Deep Attention-guided Subsampling (DAS), a framework that performs input-adaptive, differentiable subsampling of frames or slices. DAS integrates a lightweight feature extractor, a multi-head attention layer, and a differentiable Gumbel-Softmax sampling mechanism with adaptive temperature to select a subset of informative frames during inference. Through extensive experiments on eight datasets, DAS generally outperforms prior learnable subsampling methods and can approach full-sequence performance while reducing computation, with notable robustness in real-world noisy clinical data. The approach also yields an interpretable sampling matrix, and future work points to learnable feature extraction to further enhance saliency-guided sampling across broader tasks.

Abstract

Although deep neural networks have provided impressive gains in performance, these improvements often come at the cost of increased computational complexity and expense. In many cases, such as 3D volume or video classification tasks, not all slices or frames are necessary due to inherent redundancies. To address this issue, we propose a novel learnable subsampling framework that can be integrated into any neural network architecture. Subsampling, being a nondifferentiable operation, poses significant challenges for direct adaptation into deep learning models. While some works, have proposed solutions using the Gumbel-max trick to overcome the problem of non-differentiability, they fall short in a crucial aspect: they are only task-adaptive and not inputadaptive. Once the sampling mechanism is learned, it remains static and does not adjust to different inputs, making it unsuitable for real-world applications. To this end, we propose an attention-guided sampling module that adapts to inputs even during inference. This dynamic adaptation results in performance gains and reduces complexity in deep neural network models. We demonstrate the effectiveness of our method on 3D medical imaging datasets from MedMNIST3D as well as two ultrasound video datasets for classification tasks, one of them being a challenging in-house dataset collected under real-world clinical conditions.
Paper Structure (10 sections, 5 equations, 2 figures, 2 tables)

This paper contains 10 sections, 5 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Lightweight Feature Extraction
  5. Multi-Head Attention Layer
  6. Differentiable Gumbel-Softmax Sampling
  7. Experiments and Results
  8. Experimental Setup
  9. Results
  10. Conclusion

Figures (2)

  • Figure 1: Our proposed Sampling Module can be plugged into any downstream task
  • Figure 2: An overview of the proposed deep attention-guided subsampling framework. The model takes a sequence of frames, extracts features, and uses an attention mechanism to generate a sampling matrix. This matrix is then used to select a subset of frames.