SMILe: Leveraging Submodular Mutual Information For Robust Few-Shot Object Detection

TL;DR

A novel Submodular Mutual Information Learning (SMILe) framework which adopts combinatorial mutual information functions to enforce the creation of tighter and discriminative feature clusters in FSOD, agnostic of the backbone architecture demonstrating elevated performance gains.

Abstract

Confusion and forgetting of object classes have been challenges of prime interest in Few-Shot Object Detection (FSOD). To overcome these pitfalls in metric learning based FSOD techniques, we introduce a novel Submodular Mutual Information Learning (SMILe) framework which adopts combinatorial mutual information functions to enforce the creation of tighter and discriminative feature clusters in FSOD. Our proposed approach generalizes to several existing approaches in FSOD, agnostic of the backbone architecture demonstrating elevated performance gains. A paradigm shift from instance based objective functions to combinatorial objectives in SMILe naturally preserves the diversity within an object class resulting in reduced forgetting when subjected to few training examples. Furthermore, the application of mutual information between the already learnt (base) and newly added (novel) objects ensures sufficient separation between base and novel classes, minimizing the effect of class confusion. Experiments on popular FSOD benchmarks, PASCAL-VOC and MS-COCO show that our approach generalizes to State-of-the-Art (SoTA) approaches improving their novel class performance by up to 5.7% (3.3 mAP points) and 5.4% (2.6 mAP points) on the 10-shot setting of VOC (split 3) and 30-shot setting of COCO datasets respectively. Our experiments also demonstrate better retention of base class performance and up to 2x faster convergence over existing approaches agnostic of the underlying architecture.

Figures (5)

• Figure 1: Functionality of components in $L_{comb}$ proposed in the SMILe (ours) framework, (a) $L_{comb}^{inter}$ promotes separation between $C_b$ and $C_n$ while (c) $L_{comb}^{intra}$ promotes intra-class compactness.
• Figure 2: Resilience to Catastrophic forgetting and faster convergence in SMILe over SoTA approaches. (a) shows that combinatorial losses in SMILe are robust to catastrophic forgetting, while (b) shows that objectives in SMILe results in faster convergence over SoTA FSOD methods (AGCM and DiGeo).
• Figure 3: Overview of our SMILe framework highlighting the application of Mutual Information function based objectives in SMILe for the fine-tuning stage of Few-Shot Object Detection.
• Figure 4: Ablation on Overcoming Class Confusion in SMILe. (a,b) SMILe demonstrates 11% lower confusion over AGCM and (c,d) 4% lower confusion over DiGeo. Only significant numbers are highlighted. Best viewed in 200% zoom.
• Figure 5: Qualitative results from SMILe: We contrast the performance of AGCM and FSCE before and after introduction of the Combinatorial formulation introduced in SMILe. We observe significant confusion and forgetting in SoTA approaches FSCE and AGCM while introduction of SMILe overcomes most of these pitfalls.