Crowding Out The Noise: Algorithmic Collective Action Under Differential Privacy

TL;DR

This work analyzes how privacy-preserving training via DPSGD interacts with Algorithmic Collective Action (ACA). It presents a theoretical extension of ACA to DP settings, deriving bounds on the collective's ability to steer learning as a function of collective size $\alpha$, clipping $C$, noise scale $\sigma$, and privacy budget $\epsilon$, and validates these insights with experiments on MNIST, CIFAR-10, and Bank Marketing. The results show that stronger privacy (lower $\epsilon$) raises the required critical mass for successful influence, illustrating a clear privacy-ACA trade-off, while empirical privacy assessments via LiRA indicate that collective presence can modulate privacy leakage in nuanced ways. Overall, the paper highlights important societal and practical implications of deploying privacy-preserving ML, informing how privacy, trust, and collective governance may co-evolve in real-world systems.

Abstract

The integration of AI into daily life has generated considerable attention and excitement, while also raising concerns about automating algorithmic harms and re-entrenching existing social inequities. While the responsible deployment of trustworthy AI systems is a worthy goal, there are many possible ways to realize it, from policy and regulation to improved algorithm design and evaluation. In fact, since AI trains on social data, there is even a possibility for everyday users, citizens, or workers to directly steer its behavior through Algorithmic Collective Action, by deliberately modifying the data they share with a platform to drive its learning process in their favor. This paper considers how these grassroots efforts to influence AI interact with methods already used by AI firms and governments to improve model trustworthiness. In particular, we focus on the setting where the AI firm deploys a differentially private model, motivated by the growing regulatory focus on privacy and data protection. We investigate how the use of Differentially Private Stochastic Gradient Descent (DPSGD) affects the collective's ability to influence the learning process. Our findings show that while differential privacy contributes to the protection of individual data, it introduces challenges for effective algorithmic collective action. We characterize lower bounds on the success of algorithmic collective action under differential privacy as a function of the collective's size and the firm's privacy parameters, and verify these trends experimentally by simulating collective action during the training of deep neural network classifiers across several datasets.

Figures (6)

• Figure 1: The success of the collective across $\epsilon$. The top row shows results on the MNIST dataset, while the bottom one on CIFAR-10. Each column corresponds to a different clipping threshold. For each plot, we evaluate the collective's success under different values of privacy budget $\epsilon$ and compare it with the baseline case ($\epsilon = \infty, C = \infty$), which corresponds to SGD without any privacy constraints. Collective size $\alpha \in [0, 1]$ is reported as a percentage of the overall training dataset.
• Figure 2: Success of the collective across $\epsilon$ on Bank Marketing dataset uci2014bank_marketing_222. We evaluate collective's success under different values of privacy loss $\epsilon$ and compare it with baseline case ($\epsilon = \infty, C = \infty$), which corresponds to SGD without any privacy constraints.
• Figure 3: Success rate of Likelihood Ratio Attack (LiRA) carlini2022membershipinferenceattacksprinciples evaluation on CIFAR-10 dataset. Each figure corresponds to a different setting of privacy constraints with privacy increasing from left to right.
• Figure 4: Success rate of Likelihood Ratio Attack (LiRA) carlini2022membershipinferenceattacksprinciples evaluation on SVHN dataset. Each figure corresponds to a different setting of privacy constraints with privacy increasing from left to right.
• Figure 5: MNIST samples with and without adding application of transformation $g$

Theorems & Definitions (5)

• Definition 1: Gradient-redirecting distribution from hardt2024aca
• Theorem 1: Theorem 10 from hardt2024aca
• Definition 2: ($\epsilon, \delta$)-Differential Privacy
• Theorem 2
• Lemma 1