Bridging the Projection Gap: Overcoming Projection Bias Through Parameterized Distance Learning

TL;DR

The VAEGAN architecture is extended with two branches to separately output the projection of samples from seen and unseen classes, enabling more robust distance learning and introducing a novel loss function to optimize the Mahalanobis distance representation and reduce projection bias.

Abstract

Generalized zero-shot learning (GZSL) aims to recognize samples from both seen and unseen classes using only seen class samples for training. However, GZSL methods are prone to bias towards seen classes during inference due to the projection function being learned from seen classes. Most methods focus on learning an accurate projection, but bias in the projection is inevitable. We address this projection bias by proposing to learn a parameterized Mahalanobis distance metric for robust inference. Our key insight is that the distance computation during inference is critical, even with a biased projection. We make two main contributions - (1) We extend the VAEGAN (Variational Autoencoder \& Generative Adversarial Networks) architecture with two branches to separately output the projection of samples from seen and unseen classes, enabling more robust distance learning. (2) We introduce a novel loss function to optimize the Mahalanobis distance representation and reduce projection bias. Extensive experiments on four datasets show that our approach outperforms state-of-the-art GZSL techniques with improvements of up to 3.5 \% on the harmonic mean metric.

Figures (9)

• Figure 1: Demonstration of how the Mahalanobis distance compensates for the biased nature of GZSL in the projection space: when image instances and class descriptions are biased in the projection space, an image from the Cat class (indicated by the green triangle) will be misclassified into the Lion class (deep yellow pentagram) according to the Euclidean distance (the top part); however, the image will be correctly classified into the Cat class according to the Mahalanobis distance (the bottom part).
• Figure 2: Framework of VAEGAN with Mahalanobis distance, featuring two branches: the upper branch generates images of unseen classes from seen classes using a generative network to simulate classification of unseen class images during the inference phase; the lower branch directly learns the projective representations of seen class images. The newly proposed Mahalanobis distance-based loss function aims to minimize the distance between projection outputs within the same branch while maximizing the distance between projections from different branches.
• Figure 3: Performance of various $\lambda_{ \textrm{VAE} }$, $\lambda_{ \textrm{MSE} }$ and $\lambda_{ \textrm{M} }$ ratios on CUB dataset (above) and AWA2 dataset (below).
• Figure 4: Affine Transformation Fusion schematic diagram
• Figure 5: Impact of changes in the visual dimensions relative to the semantic dimensions of the dataset on model performance.