Algorithmic Collective Action in Machine Learning
Moritz Hardt, Eric Mazumdar, Celestine Mendler-Dünner, Tijana Zrnic
TL;DR
The paper develops a principled model of algorithmic collective action on ML platforms, where a small fraction $\alpha$ of participants coordinates data edits to steer the firm’s learning on the mixed distribution $\mathcal{P}=\alpha\mathcal{P}^*+(1-\alpha)\mathcal{P}_0$. It analyzes three learning contexts—classification, convex risk minimization, and gradient-based learning—proposing strategies such as signal planting, signal erasing, gradient neutralization, and gradient control, and derives quantitative thresholds for the collective size needed to achieve targets. The authors validate the theory with an extensive resume classification experiment using a BERT-like model, showing that collectives with vanishing $\alpha$ can command substantial influence, with the threshold depending on signal uniqueness, suboptimality, and training dynamics. The results illuminate practical risks and policy implications for ML powered platforms, revealing how high dimensionality and weak signals can amplify the leverage of small collectives while also guiding defenses against data manipulation attacks.
Abstract
We initiate a principled study of algorithmic collective action on digital platforms that deploy machine learning algorithms. We propose a simple theoretical model of a collective interacting with a firm's learning algorithm. The collective pools the data of participating individuals and executes an algorithmic strategy by instructing participants how to modify their own data to achieve a collective goal. We investigate the consequences of this model in three fundamental learning-theoretic settings: the case of a nonparametric optimal learning algorithm, a parametric risk minimizer, and gradient-based optimization. In each setting, we come up with coordinated algorithmic strategies and characterize natural success criteria as a function of the collective's size. Complementing our theory, we conduct systematic experiments on a skill classification task involving tens of thousands of resumes from a gig platform for freelancers. Through more than two thousand model training runs of a BERT-like language model, we see a striking correspondence emerge between our empirical observations and the predictions made by our theory. Taken together, our theory and experiments broadly support the conclusion that algorithmic collectives of exceedingly small fractional size can exert significant control over a platform's learning algorithm.