Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Biased Binary Attribute Classifiers Ignore the Majority Classes

Xinyi Zhang, Johanna Sophie Bieri, Manuel Günther

Abstract

To visualize the regions of interest that classifiers base their decisions on, different Class Activation Mapping (CAM) methods have been developed. However, all of these techniques target categorical classifiers only, though most real-world tasks are binary classification. In this paper, we extend gradient-based CAM techniques to work with binary classifiers and visualize the active regions for binary facial attribute classifiers. When training an unbalanced binary classifier on an imbalanced dataset, it is well-known that the majority class, i.e. the class with many training samples, is mostly predicted much better than minority class with few training instances. In our experiments on the CelebA dataset, we verify these results, when training an unbalanced classifier to extract 40 facial attributes simultaneously. One would expect that the biased classifier has learned to extract features mainly for the majority classes and that the proportional energy of the activations mainly reside in certain specific regions of the image where the attribute is located. However, we find very little regular activation for samples of majority classes, while the active regions for minority classes seem mostly reasonable and overlap with our expectations. These results suggest that biased classifiers mainly rely on bias activation for majority classes. When training a balanced classifier on the imbalanced data by employing attribute-specific class weights, majority and minority classes are classified similarly well and show expected activations for almost all attributes

Biased Binary Attribute Classifiers Ignore the Majority Classes

Abstract

To visualize the regions of interest that classifiers base their decisions on, different Class Activation Mapping (CAM) methods have been developed. However, all of these techniques target categorical classifiers only, though most real-world tasks are binary classification. In this paper, we extend gradient-based CAM techniques to work with binary classifiers and visualize the active regions for binary facial attribute classifiers. When training an unbalanced binary classifier on an imbalanced dataset, it is well-known that the majority class, i.e. the class with many training samples, is mostly predicted much better than minority class with few training instances. In our experiments on the CelebA dataset, we verify these results, when training an unbalanced classifier to extract 40 facial attributes simultaneously. One would expect that the biased classifier has learned to extract features mainly for the majority classes and that the proportional energy of the activations mainly reside in certain specific regions of the image where the attribute is located. However, we find very little regular activation for samples of majority classes, while the active regions for minority classes seem mostly reasonable and overlap with our expectations. These results suggest that biased classifiers mainly rely on bias activation for majority classes. When training a balanced classifier on the imbalanced data by employing attribute-specific class weights, majority and minority classes are classified similarly well and show expected activations for almost all attributes
Paper Structure (20 sections, 9 equations, 9 figures, 2 tables)

This paper contains 20 sections, 9 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Facial Attribute Classification
  4. Class Activation Mapping
  5. Approach
  6. Visualizing Binary Classifiers
  7. Balancing Facial Attribute Classifiers
  8. Selecting Frontal Test Images
  9. Evaluation Metrics
  10. Experiments
  11. Classification Errors
  12. Visualization
  13. Proportional Energy
  14. CAM Techniques
  15. Target Classes
  16. ...and 5 more sections

Figures (9)

  • Figure 1: fig:attributes Distribution of Attributes. This figure shows the distribution of the binary facial attributes throughout the CelebA dataset, indicating its large imbalance for some attributes (replicated from rudd2016moon).
  • Figure 2: fig:grad-cam-activation Averaged Grad-CAM Activations. This figure displays the average CAM activations for 15 different attributes including the probability of positive label $p_m$. Activations are averaged across all negative (left) and positive (right) predictions, extracted by AFFACT-u (top) and AFFACT-b (bottom).
  • Figure 3: fig:combined Averaged Activations for Different CAM Techniques. This figure shows the average activations for Grad-CAM (GC), GradCAM++ (GC++), HiResCAM (HR) and element-wise CAM (EW). Blocks are built identically to Figure \ref{['fig:grad-cam-activation']}.
  • Figure 4: fig:targetclass AFFACT-b Negative Class Visualization. This figure shows Grad-CAM visualizations of samples for four different attributes that were negatively predicted by the balanced network. On the left of each block, we visualize the categorical target via \ref{['eq:gradcam-categorical']}, i. e., the positive class. On the right we present the visualization of the predicted negative class created with \ref{['eq:gradcam-binary']}.
  • Figure 5: fig:masks Attribute Masks. The images in \ref{['fig:masks:masks']} show the different defined masks, applied to one input image. \ref{['fig:masks:attributes']} lists the attributes for which the masks are valid for. The last three masks are defined for single attributes.
  • ...and 4 more figures