Characterising Bias in Compressed Models

TL;DR

This work reveals that neural network compression via pruning and quantization preserves aggregate accuracy while disproportionately harming a small subset of instances, exposing or amplifying algorithmic bias. It introduces Compression Identified Exemplars (CIE) as an attribute-agnostic tool to surface challenging cases for human auditing, even when protected attribute labels are scarce. By defining Modal CIE and Taxicab CIE, the authors provide unsupervised means to rank and surface CIEs, showing these exemplars are substantially harder for both compressed and non-compressed systems. The findings argue for integrating CIE-based auditing into deployment pipelines to mitigate fairness risks in resource-constrained settings, with practical implications for domains like hiring, healthcare, and security.

Abstract

The popularity and widespread use of pruning and quantization is driven by the severe resource constraints of deploying deep neural networks to environments with strict latency, memory and energy requirements. These techniques achieve high levels of compression with negligible impact on top-line metrics (top-1 and top-5 accuracy). However, overall accuracy hides disproportionately high errors on a small subset of examples; we call this subset Compression Identified Exemplars (CIE). We further establish that for CIE examples, compression amplifies existing algorithmic bias. Pruning disproportionately impacts performance on underrepresented features, which often coincides with considerations of fairness. Given that CIE is a relatively small subset but a great contributor of error in the model, we propose its use as a human-in-the-loop auditing tool to surface a tractable subset of the dataset for further inspection or annotation by a domain expert. We provide qualitative and quantitative support that CIE surfaces the most challenging examples in the data distribution for human-in-the-loop auditing.

Figures (4)

• Figure 1: Most natural image datasets exhibit a long-tail distribution with an unequal frequency of attributes in the training data. Below each attribute sub-group in CelebA, we report the share of training set and total frequency count.
• Figure 2: Plot of the fraction of the training set of each attribute in CelebA against the relative representation of each attribute in CIE$_p$. CIE$_p$ over-index on underrepresented attributes in the dataset. In this plot we threshold Taxicab CIE generated from a pruned model at $80 \%$.
• Figure 3: Compression Identified Exemplars (CIEs) are images where there is a high level of disagreement between the predictions of pruned and non-pruned models. Visualized are a sample of CelebA CIEs alongside a non-CIE image from the same class. Above each image pair is the true label. We train a ResNet-18 on CelebA to predict a binary task of whether the hair color is blond or non-blond.
• Figure 4: Right: A comparison of model performance on $1$) a sample of Modal CIEs against the, $2$) the entire test-set and $3$) a sample excluding CIEs. Evaluation on CIE images alone yields substantially lower top-1 accuracy, Left: Comparison of non-compressed test-set accuracy (solid lines) against compressed $t=99$ pruned test-set accuracy (dashed lines) on 1) the entire test-set, with 2) Modal CIE identified at $99 \%$ pruning and 3) Taxicab CIE thresholded at different percentiles (x-axis). Any ties for Taxicab CIE are broken at random. Images with high Taxicab CIE scores and or classified as Modal CIE are far more challenging for both the non-compressed and compressed model to classify.