Exploring Cross-Domain Few-Shot Classification via Frequency-Aware Prompting

TL;DR

This paper makes one of the first attempts to propose a Frequency-Aware Prompting method with mutual attention for Cross-Domain Few-Shot classification, which can let networks simulate the human visual perception of selecting different frequency cues when facing new recognition tasks.

Abstract

Cross-Domain Few-Shot Learning has witnessed great stride with the development of meta-learning. However, most existing methods pay more attention to learning domain-adaptive inductive bias (meta-knowledge) through feature-wise manipulation or task diversity improvement while neglecting the phenomenon that deep networks tend to rely more on high-frequency cues to make the classification decision, which thus degenerates the robustness of learned inductive bias since high-frequency information is vulnerable and easy to be disturbed by noisy information. Hence in this paper, we make one of the first attempts to propose a Frequency-Aware Prompting method with mutual attention for Cross-Domain Few-Shot classification, which can let networks simulate the human visual perception of selecting different frequency cues when facing new recognition tasks. Specifically, a frequency-aware prompting mechanism is first proposed, in which high-frequency components of the decomposed source image are switched either with normal distribution sampling or zeroing to get frequency-aware augment samples. Then, a mutual attention module is designed to learn generalizable inductive bias under CD-FSL settings. More importantly, the proposed method is a plug-and-play module that can be directly applied to most off-the-shelf CD-FLS methods. Experimental results on CD-FSL benchmarks demonstrate the effectiveness of our proposed method as well as robustly improve the performance of existing CD-FLS methods. Resources at https://github.com/tinkez/FAP_CDFSC.

Figures (5)

• Figure 1: Comparison of existing CD-FSL methods with ours. (a) Feature-wise Manipulation methods have changed the backbone while (b) Task-diversity Improvement methods as well as (c) ours haven't. Thus, (c) is compatible with (a) and (b).
• Figure 2: The overall pipeline of the proposed Frequency-Aware Prompting method. $DWT$ represents the Discrete Wavelet Transform, $IDWT$ represents the Inverse Discrete Wavelet Transform, $N(0,1)$ represents the Normal Distribution Sampling, $[0,0,0]$ represents zeroing, $P(.)$ represents the network prediction confidences, and $KL\underline{~} div$ represents the KL divergence loss. An illustration of the Mutual Attention Module is also provided. Detailed explanations of our proposed Frequency-Aware Augment Samples and Mutual Attention are described in subsections 3.3 and 3.4 respectively.
• Figure 3: Average accuracy on 5-way 5-shot/1-shot tasks over eight CD-FSL datasets on three baselines with the varying learning rate $\beta$ of gradient ascending process used in the adversarial training. Other parameters are fixed as $T_{max} =5$ and $p = 0.5$ for all stages.
• Figure 4: Average accuracy on 5-way 5-shot/1-shot tasks over eight CD-FSL datasets on three baselines. It respectively shows the results of the original baseline, without mutual attention modules (w/o att) and our complete Frequency-Aware Prompting method (w att).
• Figure 5: The t-SNE visualization. (a), (b) denote mini-ImageNet while (c), (d) represent CropDisease dataset.