Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sparsity Is Necessary: Polynomial-Time Stability for Agentic LLMs in Large Action Spaces

Angshul Majumdar

TL;DR

It is shown that under partial observability, LLMs matter only through a belief/representation error epsilon_b, yielding an additive O(epsilon_b) degradation while preserving logarithmic dependence on M.

Abstract

Tool-augmented LLM systems expose a control regime that learning theory has largely ignored: sequential decision-making with a massive discrete action universe (tools, APIs, documents) in which only a small, unknown subset is relevant for any fixed task distribution. We formalize this setting as Sparse Agentic Control (SAC), where policies admit block-sparse representations over M >> 1 actions and rewards depend on sparse main effects and (optionally) sparse synergies. We study ell_{1,2}-regularized policy learning through a convex surrogate and establish sharp, compressed-sensing-style results: (i) estimation and value suboptimality scale as k (log M / T)^{1/2} under a Policy-RSC condition; (ii) exact tool-support recovery holds via primal-dual witness arguments when T > k log M under incoherence and beta-min; and (iii) any dense policy class requires Omega(M) samples, explaining the instability of prompt-only controllers. We further show that under partial observability, LLMs matter only through a belief/representation error epsilon_b, yielding an additive O(epsilon_b) degradation while preserving logarithmic dependence on M. Extensions cover tuning-free, online, robust, group-sparse, and interaction-aware SAC.

Sparsity Is Necessary: Polynomial-Time Stability for Agentic LLMs in Large Action Spaces

TL;DR

It is shown that under partial observability, LLMs matter only through a belief/representation error epsilon_b, yielding an additive O(epsilon_b) degradation while preserving logarithmic dependence on M.

Abstract

Tool-augmented LLM systems expose a control regime that learning theory has largely ignored: sequential decision-making with a massive discrete action universe (tools, APIs, documents) in which only a small, unknown subset is relevant for any fixed task distribution. We formalize this setting as Sparse Agentic Control (SAC), where policies admit block-sparse representations over M >> 1 actions and rewards depend on sparse main effects and (optionally) sparse synergies. We study ell_{1,2}-regularized policy learning through a convex surrogate and establish sharp, compressed-sensing-style results: (i) estimation and value suboptimality scale as k (log M / T)^{1/2} under a Policy-RSC condition; (ii) exact tool-support recovery holds via primal-dual witness arguments when T > k log M under incoherence and beta-min; and (iii) any dense policy class requires Omega(M) samples, explaining the instability of prompt-only controllers. We further show that under partial observability, LLMs matter only through a belief/representation error epsilon_b, yielding an additive O(epsilon_b) degradation while preserving logarithmic dependence on M. Extensions cover tuning-free, online, robust, group-sparse, and interaction-aware SAC.
Paper Structure (52 sections, 18 theorems, 109 equations)

This paper contains 52 sections, 18 theorems, 109 equations.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Why this is not an "agent" paper.
  4. Organization.
  5. Sparse Agentic Control (SAC): Problem Formulation
  6. A large tool universe and set-valued decisions
  7. Why this matters for agents.
  8. Latent-state dynamics and observations: MDP/POMDP view
  9. LLM interpretation (without baking language into the model).
  10. The sparsity hypothesis: only a few tools matter
  11. Agent meaning.
  12. A sparse parametric policy and a convex learning objective
  13. Learning from trajectories.
  14. The $\ell_{1,2}$-regularized SAC learner
  15. Interpretation for LLM agents.
  16. ...and 37 more sections

Key Result

Lemma 4.1

Assume $\widehat{\mathcal{L}}_T$ is convex and differentiable. Let $\hat{\theta}$ be any minimizer of eq:l1-learner, and set $\Delta := \hat{\theta}-\theta^\star$ with $S^\star=\mathrm{supp}(\theta^\star)$ as in eq:theta-star-support. If then $\Delta \in \mathcal{C}(S^\star)$, i.e.

Theorems & Definitions (36)

  • Lemma 4.1: Basic inequality and cone constraint
  • proof
  • Corollary 4.2: A convenient choice of $\lambda$
  • proof
  • Theorem 4.3: Estimation error under Policy-RSC
  • proof
  • Corollary 4.4: Sample complexity for accurate sparse routing
  • proof
  • Lemma 4.5: Local Hessian stability $\Rightarrow$ empirical irrepresentability
  • proof
  • ...and 26 more