Robust Few-Shot Ensemble Learning with Focal Diversity-Based Pruning

TL;DR

FusionShot tackles robustness and generalization in few-shot learning by constructing ensembles of pre-trained FS models across three fusion channels and pruning them with a focal-diversity criterion. A GA-accelerated pruning step selects compact, high-diversity sub-ensembles, and a learn-to-combine MLP nonlinearly fuses their predictions for robust ensemble outputs. Across mini-Imagenet and CUB, FusionShot variants achieve substantial gains over SOTA FS models, demonstrate strong cross-domain adaptability, and provide active defense against PGD adversarial attacks. Reproducible results and code are made available to support practical adoption and further research.

Abstract

This paper presents FusionShot, a focal diversity optimized few-shot ensemble learning approach for boosting the robustness and generalization performance of pre-trained few-shot models. The paper makes three original contributions. First, we explore the unique characteristics of few-shot learning to ensemble multiple few-shot (FS) models by creating three alternative fusion channels. Second, we introduce the concept of focal error diversity to learn the most efficient ensemble teaming strategy, rather than assuming that an ensemble of a larger number of base models will outperform those sub-ensembles of smaller size. We develop a focal-diversity ensemble pruning method to effectively prune out the candidate ensembles with low ensemble error diversity and recommend top-$K$ FS ensembles with the highest focal error diversity. Finally, we capture the complex non-linear patterns of ensemble few-shot predictions by designing the learn-to-combine algorithm, which can learn the diverse weight assignments for robust ensemble fusion over different member models. Extensive experiments on representative few-shot benchmarks show that the top-K ensembles recommended by FusionShot can outperform the representative SOTA few-shot models on novel tasks (different distributions and unknown at training), and can prevail over existing few-shot learners in both cross-domain settings and adversarial settings. For reproducibility purposes, FusionShot trained models, results, and code are made available at https://github.com/sftekin/fusionshot

Figures (16)

• Figure 1: An overview of FusionShot approach to learning ensemble fusion.
• Figure 2: All $M=1013$ ensemble teams from $N=10$ base model pool plotted with their focal diversity scores and their validation accuracy. The colors represent the size of each team, and the dotted line represents the best-performing individual model in the pool.
• Figure 3: Performance of 1-shot 5-way ensembles (mini-Imagenet) produced by distance function fusion (a), by backbone fusion (b), and hybrid of both distance function and backbone fusion (c). The green and red texts represent the best and worst performing sets among the candidate ensemble sets of the same team size. The horizontal line is the performance of the best base model.
• Figure 4: (Top) The mean accuracy of DeepEMD at 1, 5-shot settings to benign episodes (blue) and PGD attacked episodes (red). We show the mean accuracy of FusionShot to the attacked episodes in green bars. (Bottom) Red bar: # errors made by single base model or all models in a team out of 45000 novel episodes (1-shot 5-way, mini-Imagenet). Green bars: # corrected episodes by FusionShot.
• Figure 5: FusionShot under domain shift and adversarial setting.