Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Toward Better Generalization in Few-Shot Learning through the Meta-Component Combination

Qiuhao Zeng

TL;DR

This work tackles the generalization gap in metric-based few-shot learning by introducing Meta Components Learning (MCL), which represents each classifier as a weighted combination of shared meta-components learned across episodes. An Adapted MCL (AMCL) variant further improves task-specific performance by adapting the combination scores on the support set, while an orthogonality-promoting regularizer encourages diverse, informative substructures among components. The approach yields competitive or state-of-the-art results on standard few-shot classification benchmarks and extends to regression and reinforcement learning, with ablations highlighting the importance of regularization and the optimal number of components. Overall, the method enhances task-specific adaptability and shared substructure discovery, contributing to more robust generalization in unseen tasks while remaining computationally efficient.

Abstract

In few-shot learning, classifiers are expected to generalize to unseen classes given only a small number of instances of each new class. One of the popular solutions to few-shot learning is metric-based meta-learning. However, it highly depends on the deep metric learned on seen classes, which may overfit to seen classes and fail to generalize well on unseen classes. To improve the generalization, we explore the substructures of classifiers and propose a novel meta-learning algorithm to learn each classifier as a combination of meta-components. Meta-components are learned across meta-learning episodes on seen classes and disentangled by imposing an orthogonal regularizer to promote its diversity and capture various shared substructures among different classifiers. Extensive experiments on few-shot benchmark tasks show superior performances of the proposed method.

Toward Better Generalization in Few-Shot Learning through the Meta-Component Combination

TL;DR

This work tackles the generalization gap in metric-based few-shot learning by introducing Meta Components Learning (MCL), which represents each classifier as a weighted combination of shared meta-components learned across episodes. An Adapted MCL (AMCL) variant further improves task-specific performance by adapting the combination scores on the support set, while an orthogonality-promoting regularizer encourages diverse, informative substructures among components. The approach yields competitive or state-of-the-art results on standard few-shot classification benchmarks and extends to regression and reinforcement learning, with ablations highlighting the importance of regularization and the optimal number of components. Overall, the method enhances task-specific adaptability and shared substructure discovery, contributing to more robust generalization in unseen tasks while remaining computationally efficient.

Abstract

In few-shot learning, classifiers are expected to generalize to unseen classes given only a small number of instances of each new class. One of the popular solutions to few-shot learning is metric-based meta-learning. However, it highly depends on the deep metric learned on seen classes, which may overfit to seen classes and fail to generalize well on unseen classes. To improve the generalization, we explore the substructures of classifiers and propose a novel meta-learning algorithm to learn each classifier as a combination of meta-components. Meta-components are learned across meta-learning episodes on seen classes and disentangled by imposing an orthogonal regularizer to promote its diversity and capture various shared substructures among different classifiers. Extensive experiments on few-shot benchmark tasks show superior performances of the proposed method.

Paper Structure

This paper contains 23 sections, 11 equations, 5 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Meta Component Learning
  4. Preliminary: Meta-Learning Framework for Few-shot Learning
  5. Class Summarization
  6. Meta Components
  7. Score Adaptations
  8. Disentangling Regularizer
  9. MCL and AMCL for Regression and Reinforcement Learning
  10. Regression
  11. Reinforcement Learning
  12. Experiments
  13. Few-shot Classification
  14. Implementation Details
  15. Datasets
  16. ...and 8 more sections

Figures (5)

  • Figure 1: Overview of our Meta Components Learning (MCL) in classifications. $\phi(x)$ is the instance encoder network. $\{x_1\,x_2,x_3\}$ belongs to the class $c$. We use a set-function to aggregate one class's information. Then, by combining each meta-component with the scores, MCL generates the classifiers for the class $c$. (hint: better view this figure in color mode)
  • Figure 2: Four groups of images from TieredImageNet with the highest combination scores for a particular meta-component
  • Figure 3: Pearson Correlation Coefficients between meta-components and attribute learners (classifiers).
  • Figure 4: Left: 5-shot regression. Right: 10-shot regression. AMCL fits the original sine curve better than other methods.
  • Figure 5: Total rewards of different methods the in 2D Navigation task.