From FLOPs to Footprints: The Resource Cost of Artificial Intelligence

TL;DR

The paper quantifies the material footprint of AI training by linking model FLOPs to hardware material demands through ICP-OES-based elemental analysis of an Nvidia A100 GPU. It develops a framework that maps compute budgets and lifespans to per-GPU material content, providing scenario-based estimates for GPT-4 and other frontier models. Key findings show that training-scale improvements carry large material costs, with potential for substantial reductions only through combined software and hardware efficiency gains and longer hardware lifespans. The work highlights the need to incorporate material-resource considerations into AI scalability discussions and calls for greater transparency across stakeholders to make AI development more environmentally responsible.

Abstract

As computational demands continue to rise, assessing the environmental footprint of AI requires moving beyond energy and water consumption to include the material demands of specialized hardware. This study quantifies the material footprint of AI training by linking computational workloads to physical hardware needs. The elemental composition of the Nvidia A100 SXM 40 GB graphics processing unit (GPU) was analyzed using inductively coupled plasma optical emission spectroscopy, which identified 32 elements. The results show that AI hardware consists of about 90% heavy metals and only trace amounts of precious metals. The elements copper, iron, tin, silicon, and nickel dominate the GPU composition by mass. In a multi-step methodology, we integrate these measurements with computational throughput per GPU across varying lifespans, accounting for the computational requirements of training specific AI models at different training efficiency regimes. Scenario-based analyses reveal that, depending on Model FLOPs Utilization (MFU) and hardware lifespan, training GPT-4 requires between 1,174 and 8,800 A100 GPUs, corresponding to the extraction and eventual disposal of up to 7 tons of toxic elements. Combined software and hardware optimization strategies can reduce material demands: increasing MFU from 20% to 60% lowers GPU requirements by 67%, while extending lifespan from 1 to 3 years yields comparable savings; implementing both measures together reduces GPU needs by up to 93%. Our findings highlight that incremental performance gains, such as those observed between GPT-3.5 and GPT-4, come at disproportionately high material costs. The study underscores the necessity of incorporating material resource considerations into discussions of AI scalability, emphasizing that future progress in AI must align with principles of resource efficiency and environmental responsibility.

Figures (7)

• Figure 1: Annotation of the Nvidia A100 SXM 40 GB (left picture). For the chemical analysis conducted via ICP-OES, the GPU was disassembled and categorized into four main component categories: the printed circuit board (PCB) (top center), the heatsink (bottom center), the main GPU chip (top right), and the current regulators Power-on-Package (PoP) assemblies (bottom right) (author pictures).
• Figure 2: Proportion of elements in the Nvidia A100 SXM 40GB GPU (author illustration).
• Figure 3: Elemental composition of the Nvidia A100 SXM GPU by component group: heatsink, PCB, main GPU (GPU chip + VRAM), and Power-on-Packages. Displayed are the top 15 elements by weight proportion (% of total mass) (author illustration).
• Figure 4: Estimated hardware and elemental requirement for training GPT-4 at 35% MFU across varying hardware lifespan scenarios (1 to 3 years). Results are expressed in terms of the total number of GPUs required and the total elemental mass (kg) on a logarithmic scale. Extending the hardware's operational lifespan to 2 years halves the GPU demand to 2,515 GPUs, while a 3-year lifespan reduces requirements by approximately 67% to 1,676 GPUs (author's illustration).
• Figure 5: Illustration of the relationship between Falcon, Llama 2, GPT-3.5, and GPT-4 model performance - measured across five standard benchmarks - and the corresponding GPU requirements (on log scale) for model training (author illustration).