Prototype Optimization with Neural ODE for Few-Shot Learning

TL;DR

This paper regards the gradient and its flow as meta-knowledge and then proposes a novel Neural Ordinary Differential Equation (ODE)-based meta-optimizer to optimize prototypes, called MetaNODE, which consists of two novel modules: E2GradNet and E2Solver, which aim to estimate accurate gradient flows and solve optimal prototypes in an effective and efficient manner.

Abstract

Few-Shot Learning (FSL) is a challenging task, which aims to recognize novel classes with few examples. Pre-training based methods effectively tackle the problem by pre-training a feature extractor and then performing class prediction via a cosine classifier with mean-based prototypes. Nevertheless, due to the data scarcity, the mean-based prototypes are usually biased. In this paper, we attempt to diminish the prototype bias by regarding it as a prototype optimization problem. To this end, we propose a novel prototype optimization framework to rectify prototypes, i.e., introducing a meta-optimizer to optimize prototypes. Although the existing meta-optimizers can also be adapted to our framework, they all overlook a crucial gradient bias issue, i.e., the mean-based gradient estimation is also biased on sparse data. To address this issue, in this paper, we regard the gradient and its flow as meta-knowledge and then propose a novel Neural Ordinary Differential Equation (ODE)-based meta-optimizer to optimize prototypes, called MetaNODE. Although MetaNODE has shown superior performance, it suffers from a huge computational burden. To further improve its computation efficiency, we conduct a detailed analysis on MetaNODE and then design an effective and efficient MetaNODE extension version (called E2MetaNODE). It consists of two novel modules: E2GradNet and E2Solver, which aim to estimate accurate gradient flows and solve optimal prototypes in an effective and efficient manner, respectively. Extensive experiments show that 1) our methods achieve superior performance over previous FSL methods and 2) our E2MetaNODE significantly improves computation efficiency meanwhile without performance degradation.

Figures (5)

• Figure 1: Pre-training based method estimates prototypes in an average manner, which suffers from a prototype bias issue. Prior works diminish the bias in a one-step manner (a). Our method addresses it in a meta-optimization manner (b). Here, orange and black points denote training and test samples, respectively. Orange square, star, and triangle denotes the mean-based, rectified, and real prototypes, respectively.
• Figure 2: The meta-learning based prototype optimization framework, which consists pre-training, meta-training, and meta-test phases. Among them, the pre-training phase aims to learn a feature extractor to obtain a good image representation for each image. Then, a meta-optimizer is introduced to learn to rectify prototypes for each class during meta-training phase. Finally, the meta-test is employed to perform few-shot class prediction.
• Figure 3: Illustration of our MetaNODE, which consists of a GradNet and an ODE Solver. The former aims to infer the continuous-time prototype gradient flow, and the latter accounts for solving the Neural ODE to obtain the polished prototypes. The red lines represent the computation flow of the sample $x_i \in \mathcal{S} \cup \mathcal{Q}'$. Note that the gradient estimator and the weight generator are shared with all samples $x_i$ in the set of $\mathcal{S} \cup \mathcal{Q}'$.
• Figure 4: Illustration of MetaNODE extension version (i.e., E$^2$MetaNODE), which improves its two key components, i.e., GradNet and ODE solver. For GradNet, we simply its design from theoretical perspective to an more simple effective and efficient network (i.e., E2GradNet). For ODE solver, we propose a meta-learning-based ODE solver (i.e., E2Solver) for more accurate and efficient prototype optimization.
• Figure 5: Visualization of our MetaNODE on miniImagenet.