Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Greedy Is Enough: Sparse Action Discovery in Agentic LLMs

Angshul Majumdar

TL;DR

The paper addresses decision-making in agentic systems with extremely large action spaces by positing a structured, state-dependent sparsity model where only a small subset of actions influences rewards. It introduces a greedy, block-OMP–style algorithm for sparse action discovery and proves that, under standard incoherence and coverage assumptions, the true action set can be recovered with sample complexity that scales as $\mathcal{O}(k d \log M)$. After recovering the support, the authors show that a refitted estimator yields controlled estimation error and near-optimal decisions for new latent states, with information-theoretic lower bounds demonstrating the necessity of sparsity and coverage. The results explain practical pruning practices in tool-augmented LLMs and provide a theoretical foundation for scalable action pruning in large-action systems, while outlining extensions to non-linear rewards and broader POMDP contexts.

Abstract

Modern agentic systems operate in environments with extremely large action spaces, such as tool-augmented language models with thousands of available APIs or retrieval operations. Despite this scale, empirical evidence suggests that only a small subset of actions meaningfully influences performance in a given deployment. Motivated by this observation, we study a contextual linear reward model in which action relevance is governed by a structured sparsity assumption: only a small number of actions have nonzero effects across latent states. We formulate action discovery as a block-sparse recovery problem and analyze a greedy algorithm inspired by Orthogonal Matching Pursuit. Under standard assumptions on incoherence, signal strength, and action coverage, we prove that the greedy procedure exactly recovers the relevant action set with high probability, using a number of samples that scales polynomially in the sparsity level and latent dimension, and only logarithmically in the total number of actions. We further provide estimation error guarantees for refitted parameters and show that the resulting decision rule is near-optimal for new latent states. Complementing these results, we establish information-theoretic lower bounds demonstrating that sparsity and sufficient coverage are necessary for tractability. Together, our results identify sparse action discovery as a fundamental principle underlying large-action decision-making and provide a theoretical foundation for action pruning in agentic systems.

Greedy Is Enough: Sparse Action Discovery in Agentic LLMs

TL;DR

The paper addresses decision-making in agentic systems with extremely large action spaces by positing a structured, state-dependent sparsity model where only a small subset of actions influences rewards. It introduces a greedy, block-OMP–style algorithm for sparse action discovery and proves that, under standard incoherence and coverage assumptions, the true action set can be recovered with sample complexity that scales as . After recovering the support, the authors show that a refitted estimator yields controlled estimation error and near-optimal decisions for new latent states, with information-theoretic lower bounds demonstrating the necessity of sparsity and coverage. The results explain practical pruning practices in tool-augmented LLMs and provide a theoretical foundation for scalable action pruning in large-action systems, while outlining extensions to non-linear rewards and broader POMDP contexts.

Abstract

Modern agentic systems operate in environments with extremely large action spaces, such as tool-augmented language models with thousands of available APIs or retrieval operations. Despite this scale, empirical evidence suggests that only a small subset of actions meaningfully influences performance in a given deployment. Motivated by this observation, we study a contextual linear reward model in which action relevance is governed by a structured sparsity assumption: only a small number of actions have nonzero effects across latent states. We formulate action discovery as a block-sparse recovery problem and analyze a greedy algorithm inspired by Orthogonal Matching Pursuit. Under standard assumptions on incoherence, signal strength, and action coverage, we prove that the greedy procedure exactly recovers the relevant action set with high probability, using a number of samples that scales polynomially in the sparsity level and latent dimension, and only logarithmically in the total number of actions. We further provide estimation error guarantees for refitted parameters and show that the resulting decision rule is near-optimal for new latent states. Complementing these results, we establish information-theoretic lower bounds demonstrating that sparsity and sufficient coverage are necessary for tractability. Together, our results identify sparse action discovery as a fundamental principle underlying large-action decision-making and provide a theoretical foundation for action pruning in agentic systems.
Paper Structure (60 sections, 5 theorems, 77 equations)

This paper contains 60 sections, 5 theorems, 77 equations.

Table of Contents

  1. Introduction
  2. Interpretation in Agentic LLM Systems
  3. Latent state.
  4. Actions.
  5. Reward.
  6. State-dependent sparsity.
  7. Greedy action discovery.
  8. Problem Formulation
  9. Latent State and Actions
  10. Reward Model
  11. State-Dependent Sparsity
  12. Data Collection Model
  13. Objective
  14. Greedy Sparse Action Discovery
  15. Motivation
  16. ...and 45 more sections

Key Result

Theorem 4.6

Assume Assumptions assump:sparsity (Section 2), assump:state, assump:noise, assump:coverage, assump:incoherence, and assump:signal. Run Contextual Block-OMP (Section 3) for $k$ iterations, producing $S_k$. If, in addition, the following two conditions hold: for some $\alpha>0$, then $S_k = S^\star$. Moreover, under Assumptions assump:state--assump:noise and $n_{\min}\gtrsim d\log M$, the event eq

Theorems & Definitions (5)

  • Theorem 4.6: Exact recovery of $S^\star$ by Contextual Block-OMP
  • Theorem 4.7: Refit error and decision suboptimality
  • Theorem 4.8: Lower bound without sparsity
  • Theorem 5.1: Information-theoretic lower bound: $k d \log(M/k)$
  • Theorem 5.2: Coverage lower bound