Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces

Karan Gupta, Pranav Vajreshwari, Yash Pandya, Raghav Magazine, Akshay Nambi, Ahmed Awadallah

TL;DR

ATLAS is introduced, a reinforcement finetuning framework that enables SLMs to operate effectively in large-scale toolspace environments by learning how to acquire context and how to execute actions, and rubric-based reinforcement finetuning is proposed, which decomposes task success into structured, task-aligned criteria and enables scalable training using small judge models.

Abstract

Agentic systems operating over large tool ecosystems must plan and execute long-horizon workflows under weak or non-verifiable supervision. While frontier models mitigate these challenges through scale and large context budgets, small language models (SLMs) remain brittle: eager tool loading saturates context, execution errors compound over time, and sparse rewards limit learning. We introduce ATLAS, a reinforcement finetuning framework that enables SLMs to operate effectively in large-scale toolspace environments by learning how to acquire context and how to execute actions. Our approach makes two key contributions. First, we treat context control and execution structure as learnable decisions, combining iterative tool loading with programmatic tool orchestration to bound context growth and stabilize long-horizon trajectories. Second, we propose rubric-based reinforcement finetuning, which decomposes task success into structured, task-aligned criteria and enables scalable training using small judge models. Across MCP benchmarks, these design choices yield large and consistent gains over generic RL baselines, allowing a 4B SLM to approach frontier-agent performance under far tighter parameter and context budgets.

Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces

TL;DR

ATLAS is introduced, a reinforcement finetuning framework that enables SLMs to operate effectively in large-scale toolspace environments by learning how to acquire context and how to execute actions, and rubric-based reinforcement finetuning is proposed, which decomposes task success into structured, task-aligned criteria and enables scalable training using small judge models.

Abstract

Agentic systems operating over large tool ecosystems must plan and execute long-horizon workflows under weak or non-verifiable supervision. While frontier models mitigate these challenges through scale and large context budgets, small language models (SLMs) remain brittle: eager tool loading saturates context, execution errors compound over time, and sparse rewards limit learning. We introduce ATLAS, a reinforcement finetuning framework that enables SLMs to operate effectively in large-scale toolspace environments by learning how to acquire context and how to execute actions. Our approach makes two key contributions. First, we treat context control and execution structure as learnable decisions, combining iterative tool loading with programmatic tool orchestration to bound context growth and stabilize long-horizon trajectories. Second, we propose rubric-based reinforcement finetuning, which decomposes task success into structured, task-aligned criteria and enables scalable training using small judge models. Across MCP benchmarks, these design choices yield large and consistent gains over generic RL baselines, allowing a 4B SLM to approach frontier-agent performance under far tighter parameter and context budgets.
Paper Structure (52 sections, 2 equations, 3 figures, 5 tables)

This paper contains 52 sections, 2 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. ATLAS Overview
  3. Iterative Server Loading (ISL)
  4. Iterative Tool Loading (ITL)
  5. Unified Programmatic Tool Orchestration
  6. Scaffolding for Programmatic Tool Orchestration
  7. Reinforcement Finetuning
  8. Task-Level Rubrics for Agentic RFT
  9. RFT with Rubric-Based Rewards.
  10. Scaling Judging from Frontier LLMs to SLMs.
  11. Experimental Setup
  12. MCP Tasks and Environments
  13. Models
  14. Evaluation Metrics
  15. Reinforcement Finetuning Setup
  16. ...and 37 more sections

Figures (3)

  • Figure 1: Context growth and execution structure across MCP agent designs. Traditional MCP agents incur high context costs by loading all tools upfront. ISL and ITL progressively reduce context by scoping server and tool schemas, while ITL+PTC further minimizes prompt growth by moving execution state into programmatic orchestration.
  • Figure 2: ATLAS Reinforcement Finetuning approach with Rubrics as Rewards and SLM Judge.
  • Figure 3: ATLAS reinforcement finetuning training and validation curves under ISL, ITL, and ITL+PTC, showing composite training rewards (left) and task-fulfillment-based validation performance (right).