Interpretable Few-shot Learning with Online Attribute Selection

TL;DR

This work tackles interpretability in few-shot learning by introducing an inherently interpretable framework that operates in a space of human-friendly attributes. The model comprises an attribute predictor, an episode-wise attribute selector using a differentiable Gumbel-Softmax mechanism, and an optional unknown-attribute pathway with mutual-information minimization to avoid overlap with known attributes. Key contributions include (i) a per-episode attribute selection mechanism that reduces reliance on irrelevant features, (ii) an auxiliary unknown-attribute augmentation that can be activated to close accuracy gaps, and (iii) an evaluation framework that measures not only accuracy but also decision alignment with human understanding. The approach achieves competitive accuracy on four standard FSL datasets while delivering higher interpretability and enabling human-centric intervention in the decision process.

Abstract

Few-shot learning (FSL) presents a challenging learning problem in which only a few samples are available for each class. Decision interpretation is more important in few-shot classification due to a greater chance of error compared to traditional classification. However, the majority of the previous FSL methods are black-box models. In this paper, we propose an inherently interpretable model for FSL based on human-friendly attributes. Previously, human-friendly attributes have been utilized to train models with the potential for human interaction and interpretability. However, such approaches are not directly extendible to the few-shot classification scenario. Moreover, we propose an online attribute selection mechanism to effectively filter out irrelevant attributes in each episode. The attribute selection mechanism improves accuracy and helps with interpretability by reducing the number of attributes that participate in each episode. We further propose a mechanism that automatically detects the episodes where the pool of available human-friendly attributes is insufficient, and subsequently augments it by engaging some learned unknown attributes. We demonstrate that the proposed method achieves results on par with black-box few-shot learning models on four widely used datasets. We also empirically evaluate the level of decision alignment between different models and human understanding and show that our model outperforms the comparison methods based on this criterion.

Figures (6)

• Figure 1: In each few-shot episode, we filter out irrelevant attributes by proposing an online attribute selection mechanism. This improves interpretability and few-shot classification accuracy.
• Figure 2: Few-shot classification using our framework. Top: The process of predicting and selecting relevant human-friendly attributes for the current episode. Bottom: The process of predicting unknown attributes and deciding whether the unknown attributes should participate in the current episode or not.
• Figure 3: Examples of partially visible query images together with the selected attributes. In the absence of the attributes, it is hard for human to tell the image class only based on the masked image. Providing attributes makes it easier for human to predict the image class.
• Figure 4: The Grad-cam visualization of an unknown attribute for a support set (left) and a query example (right) on CUB dataset. While heatmaps provide spacial localisation, they fail to indicate what exactly the model has focused on in a region. For example, is it beak shape, beak color, eye color? The human-friendly attributes detected by the proposed method can very well address this shortcoming.
• Figure 5: The Grad-cam visualization of four different unknown attributes on AWA dataset (top) and CUB dataset (bottom).