Progress in Artificial Intelligence and its Determinants

TL;DR

The paper addresses the problem of disentangling long‑run AI progress by constructing explicit input measures ($K_t$ for computational capital and $L_t$ for AI labor) and multiple output measures, including a novel Aggregate SOTA in ML (ASOTA) index. It develops two complementary frameworks: a macro Cobb‑Douglas production function $Y_t = A_t K_t^{\alpha} L_t^{1-\alpha}$ with $\alpha \approx 0.20$, and ML scaling laws $Y \propto C^{\alpha'}$ with $\alpha' \in [0.05,0.15]$, and shows both frameworks account for exponential growth while highlighting the dominant role of Moore's Law alongside substantial labor contribution. The study provides empirical support for a minimal macro model that aligns with micro‑level scaling laws, and introduces ASOTA as a forward‑looking composite benchmark. This work offers a data‑driven basis for forecasting AI progress and informing resource allocation, emphasizing the crucial role of human capital even as hardware efficiency drives compute‑driven gains.

Abstract

We study long-run progress in artificial intelligence in a quantitative way. Many measures, including traditional ones such as patents and publications, machine learning benchmarks, and a new Aggregate State of the Art in ML (or ASOTA) Index we have constructed from these, show exponential growth at roughly constant rates over long periods. Production of patents and publications doubles every ten years, by contrast with the growth of computing resources driven by Moore's Law, roughly a doubling every two years. We argue that the input of AI researchers is also crucial and its contribution can be objectively estimated. Consequently, we give a simple argument that explains the 5:1 relation between these two rates. We then discuss the application of this argument to different output measures and compare our analyses with predictions based on machine learning scaling laws proposed in existing literature. Our quantitative framework facilitates understanding, predicting, and modulating the development of these important technologies.

Figures (6)

• Figure 1: Capital (with the price of FLOP/sec) and labor used in the AI/ML technologies sector. [Sectoral boundaries are described in the Supplement. Variables are defined as follows: $K_\textrm{FLOP/sec}$ --- capital stock (in PFLOP/sec, accounting for depreciation); $L_\textrm{CS}$ --- labour in the CS-related occupation (in persons); $P_\textrm{FLOP/sec}$ --- price (US$ per GFLOP/sec, deflated to 2017 price level).]
• Figure 2: Aggregate State of the Art in ML Index. [The number of performance metrics is the number of ML task-dataset combinations available. The Aggregate SOTA Index measures the expansion of the number of ML task-dataset combinations and improvement in their performance metrics. It uses 8858 valid task-dataset combinations available. Computed at the daily frequency, logarithm of the Index reported, 2009 standardized to 1. Annotated increments of the Index (additionally reporting the number of combinations with an improvement, and a representative example): (0) 1, including ; (1) 1, including ; (2) 3, including ; (3) 15, including ; (4) 28, including ; (5) 10, including ; (6) 9, including ; (7) 30, including ; (8) 43, including ; (9) 11, including . ]
• Figure 3: Progress in AI/ML technologies. [Theoretical model formulated as $Y_{it} = F_{it} (K_t, L_t) = A_{it} K_t^\alpha L_t^{1-\alpha}$ in logarithms. Output elasticity parameter $\alpha$ calculated from 2017 data. Time-series data is decennial-frequency before 2000, annual after that. Means of output proxy-specific $\ln(A_{it})$ are estimated by OLS, then subtracted from the corresponding proxy series to allow for series' alignment on a common plot. Then, all series standardized to a common metric, chosen to be number of papers published annually, and vertical axis is scaled in terms of decadic (base-10) logarithm of this quantity. Goodness-of-fit measures are $R^2=0.88$ for $Y_\textrm{papers}$ with 26 observations, $R^2=0.93$ for $Y_\textrm{patents}$ with 25 observations, $R^2=0.73$ for $Y_\textrm{ASOTA}$ with 14 observations, $R^2=0.71$ for $Y_\textrm{LM}$ with 9 observations, $R^2=0.66$ for $Y_\textrm{IC}$ with 12 observations, $R^2=0.79$ for $Y_\textrm{Elo}$ with 22 observations.]
• Figure S4: Investments, capital and its depreciation for the AI/ML technologies sector. [ Variables are defined as follows: $K_\textrm{FLOP/sec}$ --- capital stock (in PFLOP/sec, accounting for depreciation); $I$ --- investments in the corresponding industry (in US$mn, deflated to 2017 price level); $\delta$ --- depreciation rate in the corresponding industry (in terms of share). ]
• Figure S5: Labor used in the AI/ML technologies sector. [Variables are defined as follows: $L_\textrm{agg}$ --- labour in the aggregate economy (in persons); $L_\textrm{CS}$ --- labour in the CS-related occupations (in persons).]