KGTN-ens: Few-Shot Image Classification with Knowledge Graph Ensembles

TL;DR

The paper tackles the challenge of data-efficient few-shot image classification by leveraging external knowledge graphs. It extends the Knowledge Graph Transfer Network (KGTN) to KGTN-ens, which processes multiple graph embeddings in parallel and ensembles their class-prototype scores, enabling cost-efficient integration of diverse knowledge sources. Across evaluations on ImageNet-FS, KGTN-ens demonstrates improved top-5 accuracy (notably with hierarchy+glove using a max-ensembling strategy) and shows Wikidata embeddings can be beneficial but are not universally superior. The approach offers a scalable way to incorporate multiple structured knowledge sources into few-shot vision tasks, with limitations related to memory complexity and potential scalability to extremely large graphs.

Abstract

We propose KGTN-ens, a framework extending the recent Knowledge Graph Transfer Network (KGTN) in order to incorporate multiple knowledge graph embeddings at a small cost. We evaluate it with different combinations of embeddings in a few-shot image classification task. We also construct a new knowledge source - Wikidata embeddings - and evaluate it with KGTN and KGTN-ens. Our approach outperforms KGTN in terms of the top-5 accuracy on the ImageNet-FS dataset for the majority of tested settings.

Figures (3)

• Figure 1: Architecture of KGTN-ens.
• Figure 2: KNGT-ens (blue) performance mean top-5 accuracy with compared to the KGTN (orange) over 5 runs. KGNT-ens uses glove and hierarchy graphs combined with the max ensembling function. Horizontal lines indicate standard deviations.
• Figure 3: Adjacency matrix distributions