Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast Adaptation with Kernel and Gradient based Meta Leaning

JuneYoung Park, MinJae Kang

TL;DR

This research suggests a more efficient approach to few-shot learning and fast task adaptation compared to existing methods and lays the foundation for establishing a new paradigm in meta-learning.

Abstract

Model Agnostic Meta Learning or MAML has become the standard for few-shot learning as a meta-learning problem. MAML is simple and can be applied to any model, as its name suggests. However, it often suffers from instability and computational inefficiency during both training and inference times. In this paper, we propose two algorithms to improve both the inner and outer loops of MAML, then pose an important question about what 'meta' learning truly is. Our first algorithm redefines the optimization problem in the function space to update the model using closed-form solutions instead of optimizing parameters through multiple gradient steps in the inner loop. In the outer loop, the second algorithm adjusts the learning of the meta-learner by assigning weights to the losses from each task of the inner loop. This method optimizes convergence during both the training and inference stages of MAML. In conclusion, our algorithms offer a new perspective on meta-learning and make significant discoveries in both theory and experiments. This research suggests a more efficient approach to few-shot learning and fast task adaptation compared to existing methods. Furthermore, it lays the foundation for establishing a new paradigm in meta-learning.

Fast Adaptation with Kernel and Gradient based Meta Leaning

TL;DR

This research suggests a more efficient approach to few-shot learning and fast task adaptation compared to existing methods and lays the foundation for establishing a new paradigm in meta-learning.

Abstract

Model Agnostic Meta Learning or MAML has become the standard for few-shot learning as a meta-learning problem. MAML is simple and can be applied to any model, as its name suggests. However, it often suffers from instability and computational inefficiency during both training and inference times. In this paper, we propose two algorithms to improve both the inner and outer loops of MAML, then pose an important question about what 'meta' learning truly is. Our first algorithm redefines the optimization problem in the function space to update the model using closed-form solutions instead of optimizing parameters through multiple gradient steps in the inner loop. In the outer loop, the second algorithm adjusts the learning of the meta-learner by assigning weights to the losses from each task of the inner loop. This method optimizes convergence during both the training and inference stages of MAML. In conclusion, our algorithms offer a new perspective on meta-learning and make significant discoveries in both theory and experiments. This research suggests a more efficient approach to few-shot learning and fast task adaptation compared to existing methods. Furthermore, it lays the foundation for establishing a new paradigm in meta-learning.

Paper Structure

This paper contains 20 sections, 9 equations, 2 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Meta-Learning
  4. Few-Shot Classification
  5. Domain Generalization
  6. Method
  7. What is Meta-Learning Problem
  8. Adaptive Meta-Learning in Functional Space
  9. Kernel-Based I-AMFS
  10. Gradient-Based O-AMFS
  11. Experimental Setup
  12. Baseline
  13. Datasets
  14. Environments
  15. Hyperparameter
  16. ...and 5 more sections

Figures (2)

  • Figure 1: Diagram of our AMFS algorithm in outer loop (O-AMFS), meta-gradient is adaptively determined based on the similarity of the gradients for each task.
  • Figure 2: Accuracy based on the number of inner loop gradient updates for the Mini-ImageNet 5-way 1-shot and 5-shot tasks. Although the number of gradient update for I-AMFS was fixed at 1, it achieves faster convergence.