Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Task-Level Contrastiveness for Cross-Domain Few-Shot Learning

Kristi Topollai, Anna Choromanska

TL;DR

This work tackles the challenge of generalizing across diverse domains in few-shot learning. It introduces task-level contrastive learning, defining auxiliary task augmentations and a contrastive loss that operates on task representations to promote unsupervised clustering of tasks. The method is designed to plug into existing meta-learning algorithms, improving domain generalization while reducing computational and memory overhead; it is demonstrated on the MetaDataset benchmark with consistent gains. By enabling domain-aware modulation and unsupervised domain routing, the approach reduces reliance on domain labels and enhances practical applicability in heterogeneous task settings.

Abstract

Few-shot classification and meta-learning methods typically struggle to generalize across diverse domains, as most approaches focus on a single dataset, failing to transfer knowledge across various seen and unseen domains. Existing solutions often suffer from low accuracy, high computational costs, and rely on restrictive assumptions. In this paper, we introduce the notion of task-level contrastiveness, a novel approach designed to address issues of existing methods. We start by introducing simple ways to define task augmentations, and thereafter define a task-level contrastive loss that encourages unsupervised clustering of task representations. Our method is lightweight and can be easily integrated within existing few-shot/meta-learning algorithms while providing significant benefits. Crucially, it leads to improved generalization and computational efficiency without requiring prior knowledge of task domains. We demonstrate the effectiveness of our approach through different experiments on the MetaDataset benchmark, where it achieves superior performance without additional complexity.

Task-Level Contrastiveness for Cross-Domain Few-Shot Learning

TL;DR

This work tackles the challenge of generalizing across diverse domains in few-shot learning. It introduces task-level contrastive learning, defining auxiliary task augmentations and a contrastive loss that operates on task representations to promote unsupervised clustering of tasks. The method is designed to plug into existing meta-learning algorithms, improving domain generalization while reducing computational and memory overhead; it is demonstrated on the MetaDataset benchmark with consistent gains. By enabling domain-aware modulation and unsupervised domain routing, the approach reduces reliance on domain labels and enhances practical applicability in heterogeneous task settings.

Abstract

Few-shot classification and meta-learning methods typically struggle to generalize across diverse domains, as most approaches focus on a single dataset, failing to transfer knowledge across various seen and unseen domains. Existing solutions often suffer from low accuracy, high computational costs, and rely on restrictive assumptions. In this paper, we introduce the notion of task-level contrastiveness, a novel approach designed to address issues of existing methods. We start by introducing simple ways to define task augmentations, and thereafter define a task-level contrastive loss that encourages unsupervised clustering of task representations. Our method is lightweight and can be easily integrated within existing few-shot/meta-learning algorithms while providing significant benefits. Crucially, it leads to improved generalization and computational efficiency without requiring prior knowledge of task domains. We demonstrate the effectiveness of our approach through different experiments on the MetaDataset benchmark, where it achieves superior performance without additional complexity.

Paper Structure

This paper contains 25 sections, 16 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Meta learning
  4. Multi-domain few-shot learning
  5. Contrastive learning
  6. Preliminaries
  7. Task-level contrastive learning
  8. Task representations
  9. Task augmentations
  10. Enhancing Domain Aware Modulation and Domain Expert Routing Mechanisms
  11. Three case studies
  12. Enhancing domain aware modulation
  13. MMAML
  14. Contrastive MMAML
  15. Improving unsupervised domain routing
  16. ...and 10 more sections

Figures (3)

  • Figure 1: The 3 proposed task augmentation strategies (a) relabeling, (b) instance augmentation , and (c) mixing.
  • Figure 2: 5 way 1 shot (a,b) and 5 shot (c,d) experiments on fungi, aircraft, birds datasets. We project the adapted parameters for MAML in (a,c) and the task representations for MAML with contrastive loss in (b,d). Unlike task representations, parameters are not a reliable router for TSA-MAML.
  • Figure 3: Cluster/domain heatmap. We plot the cluster assignment of tasks for Tri-M (left) and our con-Tri-M (right). Even in the absence of domain labels (right) we are able to get assignements that closely resemble the ones obtained in a supervised manner (left)