The Geometry of Learning Under AI Delegation

TL;DR

This work describes how AI quality deforms the basin boundary and shows that these effects are robust to noise and asymmetric trust updates, and identifies stability, not incentives or misalignment, as the central mechanism by which AI assistance can undermine long-run human performance and skill.

Abstract

As AI systems shift from tools to collaborators, a central question is how the skills of humans relying on them change over time. We study this question mathematically by modeling the joint evolution of human skill and AI delegation as a coupled dynamical system. In our model, delegation adapts to relative performance, while skill improves through use and decays under non-use; crucially, both updates arise from optimizing a single performance metric measuring expected task error. Despite this local alignment, adaptive AI use fundamentally alters the global stability structure of human skill acquisition. Beyond the high-skill equilibrium of human-only learning, the system admits a *stable* low-skill equilibrium corresponding to persistent reliance, separated by a sharp basin boundary that makes early decisions effectively irreversible under the induced dynamics. We further show that AI assistance can strictly improve short-run performance while inducing persistent long-run performance loss relative to the no-AI baseline, driven by a negative feedback between delegation and practice. We characterize how AI quality deforms the basin boundary and show that these effects are robust to noise and asymmetric trust updates. Our results identify stability, not incentives or misalignment, as the central mechanism by which AI assistance can undermine long-run human performance and skill.

Figures (6)

• Figure 1: Plots illustrating the outcomes of ODE \ref{['eq:ODE_simplifed']} and effects of AI skill, with default settings of $(\theta_a, \kappa, \Delta) = (0.5, 3, 2)$.
• Figure 2: Plots illustrating the instantaneous performance loss across time with and with AI, and how the crossing point varies with respect to AI skill $\theta_a$, with default settings of $(\theta_a, \kappa, \Delta, \theta_0, p_0) = (0.5, 3, 2, 0.4, 0.3)$.
• Figure 3: Heatmap of the probability of converging to the high-skill equilibrium $(1,0)$ for SDE \ref{['eq:SDE']} as a function of the initial state $(\theta_0,p_0)$, with default settings of $(\theta_a, \kappa, \Delta, \sigma) = (0.5, 3, 2, 0.1)$. Green indicates probability $0$, and red indicates probability $1$.
• Figure 4: Plots illustrating the closeness between the stable manifold $\psi(\cdot)$ and its approximation $\widetilde{\psi}$, with default parameter settings $(\theta_a, \kappa, \Delta) = (0.5, 3, 2)$.
• Figure 5: Plots illustrating the relationship between the basin boundary $\psi(\cdot)$ and model parameters $\theta_a, \kappa, \Delta$, with default setting $(\theta_a, \kappa, \Delta) = (0.5, 3,2)$.

Theorems & Definitions (27)

• Remark 2.1: Usability of the model
• Theorem 3.1: Equilibria of ODE \ref{['eq:ODE_simplifed']}
• Theorem 3.2: Two basins divided by saddle point
• Theorem 3.3: Effects of $\theta_a$ on $\psi$
• Theorem 3.4: Short-term gains, long-term losses
• Lemma 3.5: Negative coupling between skill and delegation
• Lemma 5.1: Expected discrete dynamics
• proof
• Theorem 5.2: Existence of a solution and convergence
• proof