Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Asymptotic Cost of Complexity

Martin W Cripps

TL;DR

This paper develops a minimax framework for learning in environments with non-finite state spaces, showing that the speed of learning is governed by the metric entropy of the state space. By linking $d_K$-based ε-covering numbers to learning bounds, it derives upper and lower bounds that pin down the rate at which the minimax risk $C_t$ decays, with the key rate scaling as $t^{-2r/(1+2r)}$ where $r=\alpha+n$ captures smoothness and dimensionality. The theory is applied to two economic problems—Allocating Effort to Tasks and Exploring a Space of Alternatives—demonstrating how state-space complexity slows or speeds up the approach to optimal decisions, especially distinguishing cases with Brownian-like complexity from smoother settings. The results also contrast finite- and infinite-dimensional state spaces, showing that information demand grows rapidly as learning slows, and provide a cohesive nonparametric benchmark that extends finite-state insights to richer environments.

Abstract

We propose a measure of learning efficiency for non-finite state spaces. We characterize the complexity of a learning problem by the metric entropy of its state space. We then describe how learning efficiency is determined by this measure of complexity. This is, then, applied to two models where agents learn high-dimensional states.

The Asymptotic Cost of Complexity

TL;DR

This paper develops a minimax framework for learning in environments with non-finite state spaces, showing that the speed of learning is governed by the metric entropy of the state space. By linking -based ε-covering numbers to learning bounds, it derives upper and lower bounds that pin down the rate at which the minimax risk decays, with the key rate scaling as where captures smoothness and dimensionality. The theory is applied to two economic problems—Allocating Effort to Tasks and Exploring a Space of Alternatives—demonstrating how state-space complexity slows or speeds up the approach to optimal decisions, especially distinguishing cases with Brownian-like complexity from smoother settings. The results also contrast finite- and infinite-dimensional state spaces, showing that information demand grows rapidly as learning slows, and provide a cohesive nonparametric benchmark that extends finite-state insights to richer environments.

Abstract

We propose a measure of learning efficiency for non-finite state spaces. We characterize the complexity of a learning problem by the metric entropy of its state space. We then describe how learning efficiency is determined by this measure of complexity. This is, then, applied to two models where agents learn high-dimensional states.
Paper Structure (13 sections, 7 theorems, 81 equations, 1 figure)

This paper contains 13 sections, 7 theorems, 81 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Examples of Learning Problems
  3. Allocating Effort to Tasks
  4. Exploring a Space of Alternatives
  5. The Model
  6. Learning Efficiency and Complexity
  7. Complexity of the State Space
  8. An Upper Bound on Learning Efficiency
  9. A Lower Bound on Learning Efficiency
  10. Learning Efficiency and Complexity
  11. Complexity and Optimal Decisions
  12. Allocating Effort to Tasks
  13. Exploring a Space of Alternatives

Key Result

Proposition 1

If $\Theta=\{\theta_1,\dots,\theta_n\}$ and $\lambda^*=\max_{ij}\lambda_{ij}$, then $\mathcal{P}_\Theta$ has learning efficiency $e^{\lambda t}$ for all $\lambda\in(\lambda^*,0)$ and $e^{\lambda^*t+o(t)} \leq C_t \leq n^2 e^{\lambda^*t+o(t)}$, where $o(t)$ is a function that satisfies $\lim_{t\right

Figures (1)

  • Figure 1: The Upper Bound on Learning Efficiency

Theorems & Definitions (11)

  • Definition 1
  • Proposition 1
  • Definition 2
  • Definition 3
  • Lemma 1: Yang99
  • Proposition 2: Yang and Barron
  • Proposition 3: Yang and Barron
  • Definition 4
  • Proposition 4
  • Proposition 5
  • ...and 1 more