How Hyper-Datafication Impacts the Sustainability Costs in Frontier AI

TL;DR

The paper analyzes how frontier AI’s dependence on immense data volumes creates sustainability challenges across environmental, social, and economic dimensions. By combining a large-scale metadata study of approximately 550,000 Hugging Face datasets, a questionnaire of 134 data workers in Kenya, language-representation analyses, and external data on data-center growth, it introduces hyper-datafication as the active creation and synthetic generation of data for model training. The findings show that data growth imposes measurable storage energy use and carbon footprints, concentrates data work in precarious conditions, and biases language representation toward English and other dominant languages, with regions in the Global South bearing disproportionate burdens. The authors propose Data PROOFS—provenance, resource-awareness, ownership, openness, frugality, and standards—as a framework to mitigate these costs and spur policy, technical, and collective action toward more sustainable data practices in frontier AI.

Abstract

Large-scale data has fuelled the success of frontier artificial intelligence (AI) models over the past decade. This expansion has relied on sustained efforts by large technology corporations to aggregate and curate internet-scale datasets. In this work, we examine the environmental, social, and economic costs of large-scale data in AI through a sustainability lens. We argue that the field is shifting from building models from data to actively creating data for building models. We characterise this transition as hyper-datafication, which marks a critical juncture for the future of frontier AI and its societal impacts. To quantify and contextualise data-related costs, we analyse approximately 550,000 datasets from the Hugging Face Hub, focusing on dataset growth, storage-related energy consumption and carbon footprint, and societal representation using language data. We complement this analysis with qualitative responses from data workers in Kenya to examine the labour involved, including direct employment by big tech corporations and exposure to graphic content. We further draw on external data sources to substantiate our findings by illustrating the global disparity in data centre infrastructure. Our analyses reveal that hyper-datafication does not merely increase resource consumption but systematically redistributes environmental burdens, labour risks, and representational harms toward the Global South, precarious data workers, and under-represented cultures. Thus, we propose Data PROOFS recommendations spanning provenance, resource awareness, ownership, openness, frugality, and standards to mitigate these costs. Our work aims to make visible the often-overlooked costs of data that underpin frontier AI and to stimulate broader debate within the research community and beyond.

Figures (10)

• Figure 1: Growth of datasets and data volume over time and download concentration on the Hugging Face Hub. Left: Monthly counts of newly created datasets (100% of datasets included). Centre: Monthly data volume added (91% of datasets included). The bars are coloured by modality. For multimodal datasets, each modality contributes proportionally to the colouring. Right: Download concentration. The plot shows total downloads (solid line) and downloads in November 2025 -- which is 30 days preceding metadata extraction -- (dashed line) against dataset rank sorted by total downloads on a logarithmic scale. The shaded region highlights the top 1% of datasets (5,543 repositories).
• Figure 2: Left: Estimated provider-side storage energy (GWh). Right: Estimated user-side storage energy (TWh), assuming that 10 percent of downloads result in three months of local storage. Right axes show human-equivalent annual emissions. To provide a sense of scale, we relate cumulative emissions to the global average annual per-capita footprint of 4.73 tCO$_2$eq owid_co2_per_capita_2023.
• Figure 3: Left: Historical (2022–2024) and projected (2024–2034) electricity use for all data centres worldwide under two scenarios: a base case reflecting current regulatory conditions and industry projections, and a lift-off case assuming stronger AI adoption enabled by faster data centre deployment as modelled by the IEA iea2024energydemandai. Shown alongside the total electricity consumption in Africa assuming a 5% annual grow iea2025electricity. Right: Regional distribution of data centre electricity demand in the base case (2020–2030), showing contributions from the US, China, Asia excluding China, Europe, and the rest of the world iea2024energydemandai.
• Figure 4: Left: Distribution of respondents across salary bands by weekly working hours. Centre: Distribution of respondents across salary bands by years of experience. Right: Monthly salary distribution by exposure level, indicating no clear relationship between exposure level and salary.
• Figure 5: Gender-disaggregated distributions of weekly working hours, monthly salary, experience, data work types, and exposure to graphic content derived from the anonymous responses to the questionnaire from 134 data workers.