The Cost of Balanced Training-Data Production in an Online Data Market
Augustin Chaintreau, Roland Maio, Juba Ziani
TL;DR
The paper investigates whether ethical interventions in online data markets—specifically, forcing balanced demographic representation in training data—can be economically sustainable. It builds a stylized, endogenized market model with sellers, buyers, and a centralized marketplace employing a Myerson-like mechanism, and analyzes Nash equilibria under baseline and fairness-intervention scenarios. A key finding is that fairness interventions can backfire in small, emerging markets but may become economically feasible as markets scale, with growth amortizing the fairness cost for all participants. The work provides formal conditions under which interventions succeed or fail and emphasizes market growth as a crucial factor enabling ethically motivated data-production policies. These results suggest that ethically oriented data markets can be viable in favorable market conditions and motivate broader modeling of data production and distribution in ethical ML frameworks.
Abstract
Many ethical issues in machine learning are connected to the training data. Online data markets are an important source of training data, facilitating both production and distribution. Recently, a trend has emerged of for-profit "ethical" participants in online data markets. This trend raises a fascinating question: Can online data markets sustainably and efficiently address ethical issues in the broader machine-learning economy? In this work, we study this question in a stylized model of an online data market. We investigate the effects of intervening in the data market to achieve balanced training-data production. The model reveals the crucial role of market conditions. In small and emerging markets, an intervention can drive the data producers out of the market, so that the cost of fairness is maximal. Yet, in large and established markets, the cost of fairness can vanish (as a fraction of overall welfare) as the market grows. Our results suggest that "ethical" online data markets can be economically feasible under favorable market conditions, and motivate more models to consider the role of data production and distribution in mediating the impacts of ethical interventions.