Some Simple Economics of AGI

Abstract

For millennia, human cognition was the primary engine of progress on Earth. As AI decouples cognition from biology, the marginal cost of measurable execution falls to zero, absorbing any labor capturable by metrics--including creative, analytical, and innovative work. The binding constraint on growth is no longer intelligence but human verification bandwidth: the capacity to validate, audit, and underwrite responsibility when execution is abundant. We model the AGI transition as the collision of two racing cost curves: an exponentially decaying Cost to Automate and a biologically bottlenecked Cost to Verify. This structural asymmetry widens a Measurability Gap between what agents can execute and what humans can afford to verify. It also drives a shift from skill-biased to measurability-biased technical change. Rents migrate to verification-grade ground truth, cryptographic provenance, and liability underwriting--the ability to insure outcomes rather than merely generate them. The current human-in-the-loop equilibrium is unstable: eroded from below as apprenticeship collapses (Missing Junior Loop) and from within as experts codify their obsolescence (Codifier's Curse). Unverified deployment becomes privately rational--a Trojan Horse externality. Unmanaged, these forces pull toward a Hollow Economy. Yet by scaling verification alongside agentic capabilities, the forces that threaten collapse become the catalyst for unbounded discovery and experimentation--an Augmented Economy. We derive a practical playbook for individuals, companies, investors, and policymakers. Today's defining challenge is not the race to deploy the most autonomous systems; it is the race to secure the foundations of their oversight. Only by scaling our bandwidth for verification alongside our capacity for execution can we ensure that the intelligence we have summoned preserves the humanity that initiated it.

Figures (3)

• Figure 1: Static Regime Map with Dynamic Pressures. Each point represents a task $i$ mapped by its cost to automate ($c_A(i)=i/K_C$) and cost to verify ($c_H(i)=w\,t_{fb}(i)/S_{nm}$).
• Figure 2: Missing Junior Loop, Experience Trap and Simulation Ladder.Nullcline diagram for the human experience stock $S_{nm}$. System Dynamics: The dashed curves (blue, orange, green) plot the $\dot S_{nm}=0$ frontiers implied by the reduced-form accumulation law $\dot S_{nm} = T_m(m_A)+T_{sim}-dS_{nm}$. For the plotted illustration, we use $T_m(m_A)=T_{m0}(1-m_A)$, so the nullcline $S_{nm}^*(m_A;T_{sim})=(T_{m0}(1-m_A)+T_{sim})/d$ slopes downward: as automation increases ($m_A\uparrow$), measurable work $T_m$ contracts, lowering the steady-state experience stock unless simulation investment ($T_{sim}$) rises.Risk Threshold: The shaded gray region marks a budget-violating zone for high-stakes verification ($c_H>B$); because $c_H(i)=w\,t_{fb}(i)/S_{nm}$, verification costs increase as $S_{nm}$ falls.The Trap (Solid Red): Under a Fixed Policy with low $T_{sim}$, the trajectory inevitably crashes into the unverified gray zone.The Ladder (Dashed Purple): Under an Adaptive Policy, governance steps up $T_{sim}$ at trigger points (vertical arrows), shifting the $\dot S_{nm}=0$ frontier upward (blue $\to$ orange $\to$ green) and allowing $S_{nm}$ to land safely above the threshold despite the decline in $T_m$.
• Figure 3: Alignment Maintenance Frontier under a Widening Measurability Gap.Maintenance Frontiers (Dashed): The dashed curves plot the instantaneous $\dot{\tau}=0$ boundaries implied by $\dot{\tau}=T_{nm}(1-\tau)-\eta\,\Delta m_+\,\tau$, yielding the steady state $\tau^\star(\Delta m)=\frac{T_{nm}}{T_{nm}+\eta\,\Delta m_+}$ (where $\Delta m_+\equiv \max\{\Delta m,0\}$). For a given steering/verification allocation $T_{nm}$, points below the corresponding dashed frontier satisfy $\dot{\tau}>0$ (maintenance dominates) and points above satisfy $\dot{\tau}<0$ (drift dominates).Alignment Trajectories (Solid): The solid curves track illustrative paths as compute scaling relentlessly widens the measurability gap over time ($\dot{\Delta m}>0$, capturing $K_C\uparrow \Rightarrow m_A\uparrow$).Passive vs. Proactive Regimes: Under a No response policy with fixed low oversight ($T_{nm}=0.1$, solid red), the system eventually crosses the frontier into the drift region, causing alignment to decline steadily. Conversely, Always high oversight ($T_{nm}=0.5$, solid green) sustains higher alignment along the same gap-widening path.Adaptive Response: The solid purple curve illustrates a dynamic intervention: once drift pressure becomes salient (at $\Delta m=0.40$), institutions execute a policy step-up to increase effective steering capacity ($T_{nm}\uparrow$). This shift---achieved either by reallocating scarce expert time or through human augmentation (tooling that increases the oversight yield of a single human hour)---produces a kinked path that safely bends toward the high-oversight regime.

Theorems & Definitions (10)

• Proposition 1: Structural asymmetry of measurability
• proof
• Remark 1: Human augmentation as effective verification bandwidth
• Proposition 2: The Missing Junior Loop
• proof
• Remark 2: The Codifier's Curse
• Proposition 3: Alignment as maintenance
• proof
• Proposition 4: Governance as primitive shifts
• proof