Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Task Attribute Distance for Few-Shot Learning: Theoretical Analysis and Applications

Minyang Hu, Hong Chang, Zong Guo, Bingpeng Ma, Shiguan Shan, Xilin Chen

TL;DR

Task Attribute Distance (TAD) provides a model-agnostic metric to quantify relatedness between training and novel tasks in few-shot learning by comparing attribute conditional distributions with total variation distance and solving a minimum-weight category matching. The authors derive generalization bounds showing how TAD governs adaptation difficulty on novel tasks and demonstrate a linear relationship between TAD distance and few-shot accuracy across multiple benchmarks, with both human- and auto-annotated attributes. They validate TAD's effectiveness for predicting task difficulty and show practical applications in data augmentation and test-time intervention, including cross-dataset scenarios using CLIP-based auto-annotation. The work offers a principled, scalable approach to assess and leverage task relatedness in FSL.

Abstract

Few-shot learning (FSL) aims to learn novel tasks with very few labeled samples by leveraging experience from \emph{related} training tasks. In this paper, we try to understand FSL by delving into two key questions: (1) How to quantify the relationship between \emph{training} and \emph{novel} tasks? (2) How does the relationship affect the \emph{adaptation difficulty} on novel tasks for different models? To answer the two questions, we introduce Task Attribute Distance (TAD) built upon attributes as a metric to quantify the task relatedness. Unlike many existing metrics, TAD is model-agnostic, making it applicable to different FSL models. Then, we utilize TAD metric to establish a theoretical connection between task relatedness and task adaptation difficulty. By deriving the generalization error bound on a novel task, we discover how TAD measures the adaptation difficulty on novel tasks for FSL models. To validate our TAD metric and theoretical findings, we conduct experiments on three benchmarks. Our experimental results confirm that TAD metric effectively quantifies the task relatedness and reflects the adaptation difficulty on novel tasks for various FSL methods, even if some of them do not learn attributes explicitly or human-annotated attributes are not available. Finally, we present two applications of the proposed TAD metric: data augmentation and test-time intervention, which further verify its effectiveness and general applicability. The source code is available at https://github.com/hu-my/TaskAttributeDistance.

Task Attribute Distance for Few-Shot Learning: Theoretical Analysis and Applications

TL;DR

Task Attribute Distance (TAD) provides a model-agnostic metric to quantify relatedness between training and novel tasks in few-shot learning by comparing attribute conditional distributions with total variation distance and solving a minimum-weight category matching. The authors derive generalization bounds showing how TAD governs adaptation difficulty on novel tasks and demonstrate a linear relationship between TAD distance and few-shot accuracy across multiple benchmarks, with both human- and auto-annotated attributes. They validate TAD's effectiveness for predicting task difficulty and show practical applications in data augmentation and test-time intervention, including cross-dataset scenarios using CLIP-based auto-annotation. The work offers a principled, scalable approach to assess and leverage task relatedness in FSL.

Abstract

Few-shot learning (FSL) aims to learn novel tasks with very few labeled samples by leveraging experience from \emph{related} training tasks. In this paper, we try to understand FSL by delving into two key questions: (1) How to quantify the relationship between \emph{training} and \emph{novel} tasks? (2) How does the relationship affect the \emph{adaptation difficulty} on novel tasks for different models? To answer the two questions, we introduce Task Attribute Distance (TAD) built upon attributes as a metric to quantify the task relatedness. Unlike many existing metrics, TAD is model-agnostic, making it applicable to different FSL models. Then, we utilize TAD metric to establish a theoretical connection between task relatedness and task adaptation difficulty. By deriving the generalization error bound on a novel task, we discover how TAD measures the adaptation difficulty on novel tasks for FSL models. To validate our TAD metric and theoretical findings, we conduct experiments on three benchmarks. Our experimental results confirm that TAD metric effectively quantifies the task relatedness and reflects the adaptation difficulty on novel tasks for various FSL methods, even if some of them do not learn attributes explicitly or human-annotated attributes are not available. Finally, we present two applications of the proposed TAD metric: data augmentation and test-time intervention, which further verify its effectiveness and general applicability. The source code is available at https://github.com/hu-my/TaskAttributeDistance.
Paper Structure (22 sections, 4 theorems, 11 equations, 10 figures, 7 tables)

This paper contains 22 sections, 4 theorems, 11 equations, 10 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Few-Shot Learning
  4. Task Relatedness and Difficulty Measure
  5. Task Attribute Distance
  6. Problem Setting
  7. Measuring Task Relatedness via Attributes
  8. Distance Computation and Approximation
  9. Theoretical Analysis on Generalization
  10. A Specific Few-Shot Learning Framework
  11. Measuring Task Adaptation Difficulty via Attributes
  12. Experiments
  13. Setups
  14. Task Attribute Distance with Human-annotated Attributes
  15. Task Attribute Distance with Auto-annotated Attributes
  16. ...and 7 more sections

Key Result

Lemma 1

Let $\mathcal{A}$ be the attribute space, $L$ be the number of attributes. Assume all attributes are independent of each other given the class label, i.e. $p(a|y) = \prod_{l=1}^Lp(a^l|y)$. For all $a_i \in \mathcal{A}$ and any two categories $y_k, y_t$, the following inequality holds: where $d_{\mathcal{A}}(y_k, y_t)$ is the distance as defined in Eq.(eq:dis_category) and $\Delta=\sum_{a_i\in\mat

Figures (10)

  • Figure 1: Different settings of Few-Shot Learning (FSL). Standard FSL focuses on constructing training and novel tasks by sampling categories within a dataset (e.g. miniImageNet). Cross-Domain Few-Shot Learning (CD-FSL) considers novel tasks sampled from different dataset (e.g. CUB or SUN).
  • Figure 2: The categories and attribute sets of three bird images. Each category can be represented as a composition of some attributes, which act as a relationship bridge between different categories.
  • Figure 3: Accuracy of APNet in terms of the average task distance. (a)-(b) 5-way 1-shot and 5-shot on CUB dataset. (c)-(d) 5-way 1-shot and 5-shot on SUN dataset. In (a)-(d), each gray point denotes the average accuracy for all points in a distance interval, and the error bar denotes the confidence interval at 95% confidence level. The red dashed line is a fitted line, which shows the tendency of these gray points.
  • Figure 4: Accuracy of different methods in terms of the average task distance. From left to right, 5-way 1-shot and 5-shot on CUB/SUN. Each point denotes the average accuracy in a distance interval.
  • Figure 5: Accuracy of different methods in terms of the average task distance with (a) all attributes, (b) removal of pattern attributes, (c) removal of shape attributes, (d) removal of texture attributes. The experiment is conduct in 5-way 5-shot setting.
  • ...and 5 more figures

Theorems & Definitions (5)

  • Lemma 1
  • Theorem 1
  • Corollary 1
  • Definition 1: $\xi$-approximation meta-mapping
  • Theorem 2