Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FSL-Rectifier: Rectify Outliers in Few-Shot Learning via Test-Time Augmentation

Yunwei Bai, Ying Kiat Tan, Shiming Chen, Yao Shu, Tsuhan Chen

TL;DR

This work addresses generalization challenges in few-shot learning by mitigating outlier effects at test time. It introduces FSL-Rectifier, a training-free framework composed of an image combiner, a neighbour selector, and an augmentor, enabling test-time sample augmentation and embedding averaging to pull representations toward class centroids. The authors provide theoretical support via margin-based generalization bounds and empirical evidence on Animals and Mammals datasets, showing consistent improvements for multiple trained FSL models. The approach leverages a pretrained image translator to generate test-class samples by blending general shape with class-specific style, and uses neighbor selection to ensure quality augmentations, resulting in practical gains without additional training data or fine-tuning. Overall, FSL-Rectifier offers a viable, analysis-backed method for reducing outlier impact in FSL without retraining models."

Abstract

Few-shot learning (FSL) commonly requires a model to identify images (queries) that belong to classes unseen during training, based on a few labelled samples of the new classes (support set) as reference. So far, plenty of algorithms involve training data augmentation to improve the generalization capability of FSL models, but outlier queries or support images during inference can still pose great generalization challenges. In this work, to reduce the bias caused by the outlier samples, we generate additional test-class samples by combining original samples with suitable train-class samples via a generative image combiner. Then, we obtain averaged features via an augmentor, which leads to more typical representations through the averaging. We experimentally and theoretically demonstrate the effectiveness of our method, obtaining a test accuracy improvement proportion of around 10\% (e.g., from 46.86\% to 53.28\%) for trained FSL models. Importantly, given a pretrained image combiner, our method is training-free for off-the-shelf FSL models, whose performance can be improved without extra datasets nor further training of the models themselves. Codes are available at https://github.com/WendyBaiYunwei/FSL-Rectifier-Pub.

FSL-Rectifier: Rectify Outliers in Few-Shot Learning via Test-Time Augmentation

TL;DR

This work addresses generalization challenges in few-shot learning by mitigating outlier effects at test time. It introduces FSL-Rectifier, a training-free framework composed of an image combiner, a neighbour selector, and an augmentor, enabling test-time sample augmentation and embedding averaging to pull representations toward class centroids. The authors provide theoretical support via margin-based generalization bounds and empirical evidence on Animals and Mammals datasets, showing consistent improvements for multiple trained FSL models. The approach leverages a pretrained image translator to generate test-class samples by blending general shape with class-specific style, and uses neighbor selection to ensure quality augmentations, resulting in practical gains without additional training data or fine-tuning. Overall, FSL-Rectifier offers a viable, analysis-backed method for reducing outlier impact in FSL without retraining models."

Abstract

Few-shot learning (FSL) commonly requires a model to identify images (queries) that belong to classes unseen during training, based on a few labelled samples of the new classes (support set) as reference. So far, plenty of algorithms involve training data augmentation to improve the generalization capability of FSL models, but outlier queries or support images during inference can still pose great generalization challenges. In this work, to reduce the bias caused by the outlier samples, we generate additional test-class samples by combining original samples with suitable train-class samples via a generative image combiner. Then, we obtain averaged features via an augmentor, which leads to more typical representations through the averaging. We experimentally and theoretically demonstrate the effectiveness of our method, obtaining a test accuracy improvement proportion of around 10\% (e.g., from 46.86\% to 53.28\%) for trained FSL models. Importantly, given a pretrained image combiner, our method is training-free for off-the-shelf FSL models, whose performance can be improved without extra datasets nor further training of the models themselves. Codes are available at https://github.com/WendyBaiYunwei/FSL-Rectifier-Pub.
Paper Structure (24 sections, 2 theorems, 20 equations, 4 figures, 5 tables)

This paper contains 24 sections, 2 theorems, 20 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. FSL-Rectifier
  4. Preliminary: Few-Shot Learning
  5. Method
  6. Overall Pipeline.
  7. Image Combiner.
  8. Neighbour Selector.
  9. Augmentor.
  10. Theoretical Insights
  11. Problem Formulation
  12. Generalization Bound Analysis
  13. Implementation and Experiments
  14. Datasets
  15. Models and Training
  16. ...and 9 more sections

Key Result

Proposition 1

When the cardinality of $S_1$ is $v$, we have $\mathbb{P}_{S_1, S_2}(\max_{S_1} \|\mathbf{\Omega}\|>\|\mathbf{\Omega}'\|) = \mathbb{P}_{S_1, S_2}(r>\|\mathbf{\Omega}'\|) = 1 - \frac{1}{2^v}$, $\forall \mathbf{\Omega}' \in S_2$.

Figures (4)

  • Figure 1: Illustration of our key idea. (a): The first image in each column of the animals dataset is the original test sample, the second is neighbour sample from the test class. The last image is the generation based on style of the original test sample and general shape of the neighbour. (b): Given different neighbour samples, each test sample can be augmented to become multiple copies.
  • Figure 2: Architecture of FSL-Rectifier.
  • Figure 3: TSNE plot indicating that average augmentation of a random point P ($\star$) stay closer to the class centroid ($\blacktriangledown$), compared to the random point P ($\bullet$) on its own. Best viewed in colors.
  • Figure 4: Illustration of the effect of neighbour selector. The first row are the original test samples, the second row are neighbours picked by either neighbour selector or inverse neighbour selector. The third row are new test samples generated by the image combiner based on neighbours picked by different neighbour selectors.

Theorems & Definitions (3)

  • Proposition 1
  • Definition 1: Margin Loss Function
  • Theorem 1