Can sparsity improve the privacy of neural networks?

TL;DR

This work investigates whether sparsity in neural networks can enhance data privacy against membership inference attacks. By employing a shadow-model framework and two sparsity regimes—unstructured IMP and structured butterfly sparsity—the authors observe a positive association between sparsity, privacy, and classification error, while cautioning that accuracy confounds privacy conclusions. They argue that prior studies may misattribute privacy gains to sparsity due to uncontrolled accuracy and high result variability, advocating for experiments that fix performance levels. The findings underscore a nuanced privacy-accuracy trade-off in sparse models and call for larger-scale evaluations and stronger attacks to draw robust conclusions about sparsity’s privacy benefits.

Abstract

Sparse neural networks are mainly motivated by ressource efficiency since they use fewer parameters than their dense counterparts but still reach comparable accuracies. This article empirically investigates whether sparsity could also improve the privacy of the data used to train the networks. The experiments show positive correlations between the sparsity of the model, its privacy, and its classification error. Simply comparing the privacy of two models with different sparsity levels can yield misleading conclusions on the role of sparsity, because of the additional correlation with the classification error. From this perspective, some caveats are raised about previous works that investigate sparsity and privacy.

Figures (3)

• Figure 1: Means and standard deviations of the accuracy and defense level of various sparse networks. The percentage of nonzero weights is given in blue for IMP ($*$p$\%$), and in red for Butterfly (• p$\%$). The color indicates the percentage of nonzero parameters (in black from $50$ to $100\%$).
• Figure 2: Experiments obey to the following pipeline: two networks are trained in the same fashion on $\mathcal{D}_{\text{train}}^{\text{target}}$ and $\mathcal{D}_{\text{train}}^{\text{shadow}}$ respectively. $\mathcal{R}^{\text{shadow}}$, $\mathcal{D}_{\text{train}}^{\text{shadow}}$ and $\mathcal{D}_{\text{test}}^{\text{shadow}}$ are then used to train a discriminator that will attack $\mathcal{R}^{\text{target}}$ by trying to infer the membership of $x$ in $\mathcal{D}_{\text{train}}^{\text{target}}$.
• Figure 3: Example of supports enforced to the sparse factors in a butterfly decomposition.