Efficient DNN-Powered Software with Fair Sparse Models

TL;DR

The paper tackles fairness degradation in LTH-based pruning of DNN-powered software by introducing Ballot, a fairness-aware pruning framework that detects optimization conflicts between accuracy and fairness during subnetwork selection and applies training refinement to yield accurate and fair sparse tickets under a sparsity constraint $|\theta^*|/|\theta| \le \Omega$ with $Acc(\theta) - Acc(\theta^*) \le \varepsilon$, evaluated via fairness metrics $CWV$ and $MCD$. Ballot employs a conflict-detection-based mask generation to prune neurons that conflict with fairness goals and then refines training with a dynamic learning-rate schedule and rewind mechanisms to maximize the ticket’s performance. Across five datasets and three models, Ballot achieves substantial gains in fairness (lower CWV and MCD) while maintaining or improving accuracy compared with Magnitude Pruning, Standard LTH, SafeCompress, and FairScratch. The work demonstrates a practical pathway to deploying fairer, more efficient compressed DNN software in real-world systems, with an openly available implementation. The approach highlights the importance of integrating ethics-aware considerations into model compression to support responsible Software 3.0 deployment.

Abstract

With the emergence of the Software 3.0 era, there is a growing trend of compressing and integrating large models into software systems, with significant societal implications. Regrettably, in numerous instances, model compression techniques impact the fairness performance of these models and thus the ethical behavior of DNN-powered software. One of the most notable example is the Lottery Ticket Hypothesis (LTH), a prevailing model pruning approach. This paper demonstrates that fairness issue of LTHbased pruning arises from both its subnetwork selection and training procedures, highlighting the inadequacy of existing remedies. To address this, we propose a novel pruning framework, Ballot, which employs a novel conflict-detection-based subnetwork selection to find accurate and fair subnetworks, coupled with a refined training process to attain a high-performance model, thereby improving the fairness of DNN-powered software. By means of this procedure, Ballot improves the fairness of pruning by 38.00%, 33.91%, 17.96%, and 35.82% compared to state-of-the-art baselines, namely Magnitude Pruning, Standard LTH, SafeCompress, and FairScratch respectively, based on our evaluation of five popular datasets and three widely used models. Our code is available at https://anonymous.4open.science/r/Ballot-506E.

Figures (6)

• Figure 1: Performance of ResNet50 Gender Classifier and its sparse versions after LTH-based pruning.
• Figure 2: GradCAM heatmap for different models. (a) is the original image; (b)-(d) are the results of the original model, the model after random mask application and the model after Ballot pruning.
• Figure 3: Average model performance on CIFAR-100 dataset for different training epochs setup.
• Figure 4: Overview of Ballot.
• Figure 5: Time to prune and retrain a model.