LDCA: Local Descriptors with Contextual Augmentation for Few-Shot Learning

TL;DR

LDCA tackles the few-shot learning problem by enriching local image descriptors with global contextual information through a visual-transformer module. It combines a CNN-based feature extractor with a context augmentation stage and a cosine-based $k$-NN classifier to operate on enhanced local descriptors, enabling robust performance in 5-way $K$-shot tasks. The main contributions are the LDCA module that injects global context and positional cues into local features, the gating mechanism to boost discriminability, and empirical results showing up to 20% absolute improvement on fine-grained datasets and reduced sensitivity to the choice of $k$ in $k$-NN. These findings demonstrate the practical value of integrating local and global information in few-shot learning, with strong transferability to cross-domain fine-grained tasks.

Abstract

Few-shot image classification has emerged as a key challenge in the field of computer vision, highlighting the capability to rapidly adapt to new tasks with minimal labeled data. Existing methods predominantly rely on image-level features or local descriptors, often overlooking the holistic context surrounding these descriptors. In this work, we introduce a novel approach termed "Local Descriptor with Contextual Augmentation (LDCA)". Specifically, this method bridges the gap between local and global understanding uniquely by leveraging an adaptive global contextual enhancement module. This module incorporates a visual transformer, endowing local descriptors with contextual awareness capabilities, ranging from broad global perspectives to intricate surrounding nuances. By doing so, LDCA transcends traditional descriptor-based approaches, ensuring each local feature is interpreted within its larger visual narrative. Extensive experiments underscore the efficacy of our method, showing a maximal absolute improvement of 20\% over the next-best on fine-grained classification datasets, thus demonstrating significant advancements in few-shot classification tasks.

Figures (2)

• Figure 1: (a)The illustration of samples belongs to the same class.Feature misalignment occurs when the local descriptor of the query (highlighted in red) mistakenly associates with a similarly colored but irrelevant background region in the support image (enclosed in yellow). This is indicative of the limitations inherent in methods that rely solely on direct feature comparison without contextual consideration.(b)The illustration showcases samples from different classes and highlights the challenge of distinguishing ambiguous regions within fine-grained classification datasets. These datasets frequently contain repetitive patterns, such as texture, color, and shape, which complicate differentiation when relying solely on local information
• Figure 2: The proposed LDCA method's framework for 5-way 5-shot classification consists of three key components: (i) a feature embedding model, utilizing a CNN to extract local descriptors from images; (ii) a contextual augmentation model that adaptively integrates global context and positional information into the local descriptors of both support and query images; (iii) a k-NN based classifier that computes the similarity between query set images and each class in the support set.c$i$ represents the $i$-th class, $q$ represents the query set image