FAD: Frequency Adaptation and Diversion for Cross-domain Few-shot Learning

TL;DR

FAD tackles cross-domain few-shot learning by introducing a Frequency Diversion Adapter that transforms features into the frequency domain, partitions them into low, mid, and high bands with radial masks, and applies band-specific convolutions. This frequency-aware, band-wise modulation is integrated as a plug-and-play residual adapter that updates only during meta-testing, enabling disentangled adaptation aligned with spectral semantics. On Meta-Dataset, FAD achieves state-of-the-art performance, with pronounced gains on unseen domains and across varying-way settings, and ablations validate the benefits of spectral partitioning and kernel design. The work demonstrates that spectral representations offer a principled basis for robust generalization under distribution shift and limited supervision in cross-domain scenarios.

Abstract

Cross-domain few-shot learning (CD-FSL) requires models to generalize from limited labeled samples under significant distribution shifts. While recent methods enhance adaptability through lightweight task-specific modules, they operate solely in the spatial domain and overlook frequency-specific variations that are often critical for robust transfer. We observe that spatially similar images across domains can differ substantially in their spectral representations, with low and high frequencies capturing complementary semantic information at coarse and fine levels. This indicates that uniform spatial adaptation may overlook these spectral distinctions, thus constraining generalization. To address this, we introduce Frequency Adaptation and Diversion (FAD), a frequency-aware framework that explicitly models and modulates spectral components. At its core is the Frequency Diversion Adapter, which transforms intermediate features into the frequency domain using the discrete Fourier transform (DFT), partitions them into low, mid, and high-frequency bands via radial masks, and reconstructs each band using inverse DFT (IDFT). Each frequency band is then adapted using a dedicated convolutional branch with a kernel size tailored to its spectral scale, enabling targeted and disentangled adaptation across frequencies. Extensive experiments on the Meta-Dataset benchmark demonstrate that FAD consistently outperforms state-of-the-art methods on both seen and unseen domains, validating the utility of frequency-domain representations and band-wise adaptation for improving generalization in CD-FSL.

Figures (4)

• Figure 1: (a) Images with similar spatial semantics from different domains can differ significantly in their frequency spectra, reflecting domain-specific variations in texture, style, and structure. (b) Grad-CAM selvaraju2017grad visualizations with frequency masking reveal a shift in attention from coarse to fine regions as frequency increases, highlighting the semantic specificity of spectral bands.
• Figure 2: Overview of the Frequency Diversion Adapter. (a) Frequency Diversion transforms features into the frequency domain and partitions them into spectral bands using radial masks. (b) Band-wise Adaptation reconstructs each band via inverse DFT and modulates it using dedicated convolutional branches. This design enables fine-grained, frequency-aware adaptation for improved generalization across domains.
• Figure 3: Ablation study on block-wise adapter integration under Varying-Way Five-Shot setting on unseen domains. Adapters are inserted at different blocks of ResNet-18 to assess the contribution of spectral modulation at various feature depths.
• Figure 4: Grad-CAM visualizations for different frequency branches in FAD. For each query image, we visualize the model's attention when only one frequency branch—low, mid, or high—is activated.