Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Few-Shot Image Classification through Learnable Multi-Scale Embedding and Attention Mechanisms

Fatemeh Askari, Amirreza Fateh, Mohammad Reza Mohammadi

TL;DR

This work tackles the data scarcity challenge in few-shot image classification by introducing MSENet, a framework that builds a learnable multi-scale embedding with self-attention and per-stage fusion weights. By extracting five feature maps from a ResNet-18 backbone, refining them with self-attention, and aggregating per-stage distances via trainable weights, the method forms robust, multi-scale representations and prototypes for each class. Empirical results on MiniImageNet and FC100 show state-of-the-art 1-shot and 5-shot accuracy, with strong cross-domain generalization across eight benchmark datasets and a histopathology dataset. The study also provides extensive ablations and analyses, highlighting the contributions of multiscale features, learnable weights, and attention, and points to future directions in weight initialization theory and two-phase training for further improvements.

Abstract

In the context of few-shot classification, the goal is to train a classifier using a limited number of samples while maintaining satisfactory performance. However, traditional metric-based methods exhibit certain limitations in achieving this objective. These methods typically rely on a single distance value between the query feature and support feature, thereby overlooking the contribution of shallow features. To overcome this challenge, we propose a novel approach in this paper. Our approach involves utilizing a multi-output embedding network that maps samples into distinct feature spaces. The proposed method extracts feature vectors at different stages, enabling the model to capture both global and abstract features. By utilizing these diverse feature spaces, our model enhances its performance. Moreover, employing a self-attention mechanism improves the refinement of features at each stage, leading to even more robust representations and improved overall performance. Furthermore, assigning learnable weights to each stage significantly improved performance and results. We conducted comprehensive evaluations on the MiniImageNet and FC100 datasets, specifically in the 5-way 1-shot and 5-way 5-shot scenarios. Additionally, we performed cross-domain tasks across eight benchmark datasets, achieving high accuracy in the testing domains. These evaluations demonstrate the efficacy of our proposed method in comparison to state-of-the-art approaches. https://github.com/FatemehAskari/MSENet

Enhancing Few-Shot Image Classification through Learnable Multi-Scale Embedding and Attention Mechanisms

TL;DR

This work tackles the data scarcity challenge in few-shot image classification by introducing MSENet, a framework that builds a learnable multi-scale embedding with self-attention and per-stage fusion weights. By extracting five feature maps from a ResNet-18 backbone, refining them with self-attention, and aggregating per-stage distances via trainable weights, the method forms robust, multi-scale representations and prototypes for each class. Empirical results on MiniImageNet and FC100 show state-of-the-art 1-shot and 5-shot accuracy, with strong cross-domain generalization across eight benchmark datasets and a histopathology dataset. The study also provides extensive ablations and analyses, highlighting the contributions of multiscale features, learnable weights, and attention, and points to future directions in weight initialization theory and two-phase training for further improvements.

Abstract

In the context of few-shot classification, the goal is to train a classifier using a limited number of samples while maintaining satisfactory performance. However, traditional metric-based methods exhibit certain limitations in achieving this objective. These methods typically rely on a single distance value between the query feature and support feature, thereby overlooking the contribution of shallow features. To overcome this challenge, we propose a novel approach in this paper. Our approach involves utilizing a multi-output embedding network that maps samples into distinct feature spaces. The proposed method extracts feature vectors at different stages, enabling the model to capture both global and abstract features. By utilizing these diverse feature spaces, our model enhances its performance. Moreover, employing a self-attention mechanism improves the refinement of features at each stage, leading to even more robust representations and improved overall performance. Furthermore, assigning learnable weights to each stage significantly improved performance and results. We conducted comprehensive evaluations on the MiniImageNet and FC100 datasets, specifically in the 5-way 1-shot and 5-way 5-shot scenarios. Additionally, we performed cross-domain tasks across eight benchmark datasets, achieving high accuracy in the testing domains. These evaluations demonstrate the efficacy of our proposed method in comparison to state-of-the-art approaches. https://github.com/FatemehAskari/MSENet
Paper Structure (18 sections, 11 equations, 6 figures, 9 tables)

This paper contains 18 sections, 11 equations, 6 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Proposed method
  4. Problem definition
  5. Overview
  6. Backbone
  7. Attention module
  8. Distance metric
  9. Learnable weights
  10. Experimental Results
  11. Datasets
  12. Experimental Setting
  13. Evaluation metric
  14. Comparison with state of the arts
  15. Cross domain
  16. ...and 3 more sections

Figures (6)

  • Figure 1: The overview architecture of the proposed model
  • Figure 2: Visualization of feature maps from the five convolutional stages of ResNet-18, illustrating the progression from low-level features in shallow layers to high-level semantic features in deeper layers, critical for accurate classification.
  • Figure 3: SA module
  • Figure 4: Examples of correctly classified samples on the MiniImageNet dataset.
  • Figure 5: Examples of misclassified samples on the MiniImageNet dataset.
  • ...and 1 more figures