A Fair Loss Function for Network Pruning

TL;DR

The performance weighted loss function is introduced, a simple modified cross-entropy loss function that can be used to limit the introduction of biases during pruning.

Abstract

Model pruning can enable the deployment of neural networks in environments with resource constraints. While pruning may have a small effect on the overall performance of the model, it can exacerbate existing biases into the model such that subsets of samples see significantly degraded performance. In this paper, we introduce the performance weighted loss function, a simple modified cross-entropy loss function that can be used to limit the introduction of biases during pruning. Experiments using the CelebA, Fitzpatrick17k and CIFAR-10 datasets demonstrate that the proposed method is a simple and effective tool that can enable existing pruning methods to be used in fairness sensitive contexts. Code used to produce all experiments contained in this paper can be found at https://github.com/robbiemeyer/pw_loss_pruning.

Figures (8)

• Figure 1: Mean pruning performance with ResNet-18 and VGG-16 models with CelebA dataset. Red stars indicate degenerate trials.
• Figure 2: Mean pruning performance with ResNet-34 and EfficientNet-V2 Med. models with Fitzpatrick17k dataset.
• Figure 3: Mean pruning performance with ResNet-34 and EfficientNet-V2 Small models with CIFAR-10 dataset.
• Figure 4: Change in the proportion of the batch loss due to the use of the PW loss, segmented by class and attribute.
• Figure 5: Pruning performance with ResNet-18 models trained on subsets of CelebA dataset with alternative class and gender balances.