GRSDet: Learning to Generate Local Reverse Samples for Few-shot Object Detection

TL;DR

This work designs an intra-class feature converter (IFC) as the generator of CCVA to learn the selecting rule of LRSamples, and designs a Feature Density Boundary Optimization module to adaptively adjust the importance of samples depending on their distance from the decision boundary.

Abstract

Few-shot object detection (FSOD) aims to achieve object detection only using a few novel class training data. Most of the existing methods usually adopt a transfer-learning strategy to construct the novel class distribution by transferring the base class knowledge. However, this direct way easily results in confusion between the novel class and other similar categories in the decision space. To address the problem, we propose generating local reverse samples (LRSamples) in Prototype Reference Frames to adaptively adjust the center position and boundary range of the novel class distribution to learn more discriminative novel class samples for FSOD. Firstly, we propose a Center Calibration Variance Augmentation (CCVA) module, which contains the selection rule of LRSamples, the generator of LRSamples, and augmentation on the calibrated distribution centers. Specifically, we design an intra-class feature converter (IFC) as the generator of CCVA to learn the selecting rule. By transferring the knowledge of IFC from the base training to fine-tuning, the IFC generates plentiful novel samples to calibrate the novel class distribution. Moreover, we propose a Feature Density Boundary Optimization (FDBO) module to adaptively adjust the importance of samples depending on their distance from the decision boundary. It can emphasize the importance of the high-density area of the similar class (closer decision boundary area) and reduce the weight of the low-density area of the similar class (farther decision boundary area), thus optimizing a clearer decision boundary for each category. We conduct extensive experiments to demonstrate the effectiveness of our proposed method. Our method achieves consistent improvement on the Pascal VOC and MS COCO datasets based on DeFRCN and MFDC baselines.

Figures (8)

• Figure 1: Comparison of traditional strategy and our motivation. (a) The light orange space surrounded by dashed lines is an augmented space, while the deep orange space surrounded by solid lines is the original space. The yellow dots represent the centers of each category. (b) The solid red line represents the conversion between samples achieved through an intra-class feature converter (IFC). The red pentagram is the calibrated sample center in the novel class space. (c) The dashed line represents the decision boundary between different categories, and different colors represent different importance and reweighting functions $\mathcal{G}(\cdot)$.
• Figure 2: Overview of our proposed GRSDet. The Center Calibration Variance Augmentation (CCVA) module implements LRSamples generated through learning on base training and transferring on fine-tuning, then uses similar base class variance to augment the calibrated novel feature. The Feature Density Boundary Optimization (FDBO) module judges the distance from the decision boundary by calculating the sample density to design the weight of importance. More details can be referred to in Section \ref{['method']}.
• Figure 3: Structure of Center Calibration Variance Augmentation module. It contains the Selection Criterion of LRSamples, IFC on base training and fine-tuning stages, and variance augmentation three processes. The type of sample shape is the same as Figure \ref{['overview']}.
• Figure 4: Structure of Feature Density Boundary Optimization module. Dark orange and light orange features belong to the same category and have different importance depending on the density.
• Figure 5: Ablation of the number of generating LRSamples.