Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Unreasonable Effectiveness of Scaling Laws in AI

Chien-Ping Lu

Abstract

Classical AI scaling laws, especially for pre-training, describe how training loss decreases with compute in a power-law form. Their effectiveness has a basic and very practical sense: they make progress predictable, albeit at a declining rate. Yet their effectiveness is also unreasonable in two further senses. First, these laws are largely empirical and observational, but they appear repeatedly across model families and increasingly across training-adjacent regimes. Second, despite the diminishing returns they predict, progress in practice has often continued through rapidly improving efficiency, visible for example in falling cost per token. This paper argues that both features arise from the same source: scaling laws are unusually effective because they abstract away from many realization details. The compute variable is best understood as logical compute, an implementation-agnostic notion of model-side work, while the practical burden of scaling depends on how efficiently real resources are converted into that compute. This abstraction helps explain both why the laws travel so well across settings and why they give rise to a persistent efficiency game in hardware, algorithms, and systems. Once efficiency is made explicit, the main practical question becomes how many efficiency doublings are required to keep scaling productive despite diminishing returns. Under that view, diminishing returns are not only a geometric flattening of the loss curve, but also rising pressure for cost reduction, system-level innovation, and the breakthroughs needed to sustain Moore-like efficiency doublings.

The Unreasonable Effectiveness of Scaling Laws in AI

Abstract

Classical AI scaling laws, especially for pre-training, describe how training loss decreases with compute in a power-law form. Their effectiveness has a basic and very practical sense: they make progress predictable, albeit at a declining rate. Yet their effectiveness is also unreasonable in two further senses. First, these laws are largely empirical and observational, but they appear repeatedly across model families and increasingly across training-adjacent regimes. Second, despite the diminishing returns they predict, progress in practice has often continued through rapidly improving efficiency, visible for example in falling cost per token. This paper argues that both features arise from the same source: scaling laws are unusually effective because they abstract away from many realization details. The compute variable is best understood as logical compute, an implementation-agnostic notion of model-side work, while the practical burden of scaling depends on how efficiently real resources are converted into that compute. This abstraction helps explain both why the laws travel so well across settings and why they give rise to a persistent efficiency game in hardware, algorithms, and systems. Once efficiency is made explicit, the main practical question becomes how many efficiency doublings are required to keep scaling productive despite diminishing returns. Under that view, diminishing returns are not only a geometric flattening of the loss curve, but also rising pressure for cost reduction, system-level innovation, and the breakthroughs needed to sustain Moore-like efficiency doublings.

Paper Structure

This paper contains 7 sections, 19 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Classical Scaling Law and What It Omits
  3. Logical Compute and Hidden Efficiency
  4. A Time-Indexed Efficiency-Doubling Extension
  5. The Operational Meaning of Diminishing Returns
  6. Brief Implications
  7. Conclusion

Figures (1)

  • Figure 1: Relative excess loss over calendar time for several annual efficiency-doubling rates, holding $\kappa=0.063$ fixed. The blue curve uses a Moore-like baseline of $\beta=0.5$ doublings per year, while the dashed curve shows the non-doubling case with linear compute growth. Faster annual doubling rates accelerate the accumulation of logical compute and therefore the decline in excess loss.