Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization

Zhengxi Lu, Zhiyuan Yao, Jinyang Wu, Chengcheng Han, Qi Gu, Xunliang Cai, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen

Abstract

Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a reliable mechanism for augmenting LLM agents. Yet inference-time skill augmentation is fundamentally limited: retrieval noise introduces irrelevant guidance, injected skill content imposes substantial token overhead, and the model never truly acquires the knowledge it merely follows. We ask whether skills can instead be internalized into model parameters, enabling zero-shot autonomous behavior without any runtime skill retrieval. We introduce SKILL0, an in-context reinforcement learning framework designed for skill internalization. SKILL0 introduces a training-time curriculum that begins with full skill context and progressively withdraws it. Skills are grouped offline by category and rendered with interaction history into a compact visual context, teaching he model tool invocation and multi-turn task completion. A Dynamic Curriculum then evaluates each skill file's on-policy helpfulness, retaining only those from which the current policy still benefits within a linearly decaying budget, until the agent operates in a fully zero-shot setting. Extensive agentic experiments demonstrate that SKILL0 achieves substantial improvements over the standard RL baseline (+9.7\% for ALFWorld and +6.6\% for Search-QA), while maintaining a highly efficient context of fewer than 0.5k tokens per step. Our code is available at https://github.com/ZJU-REAL/SkillZero.

SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization

Abstract

Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a reliable mechanism for augmenting LLM agents. Yet inference-time skill augmentation is fundamentally limited: retrieval noise introduces irrelevant guidance, injected skill content imposes substantial token overhead, and the model never truly acquires the knowledge it merely follows. We ask whether skills can instead be internalized into model parameters, enabling zero-shot autonomous behavior without any runtime skill retrieval. We introduce SKILL0, an in-context reinforcement learning framework designed for skill internalization. SKILL0 introduces a training-time curriculum that begins with full skill context and progressively withdraws it. Skills are grouped offline by category and rendered with interaction history into a compact visual context, teaching he model tool invocation and multi-turn task completion. A Dynamic Curriculum then evaluates each skill file's on-policy helpfulness, retaining only those from which the current policy still benefits within a linearly decaying budget, until the agent operates in a fully zero-shot setting. Extensive agentic experiments demonstrate that SKILL0 achieves substantial improvements over the standard RL baseline (+9.7\% for ALFWorld and +6.6\% for Search-QA), while maintaining a highly efficient context of fewer than 0.5k tokens per step. Our code is available at https://github.com/ZJU-REAL/SkillZero.

Paper Structure

This paper contains 35 sections, 11 equations, 12 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM Agents
  4. Agentic Skills
  5. Method: Skill0
  6. Agent Loop
  7. Task Definition.
  8. Skill Management.
  9. Context Rendering.
  10. In-Context Reinforcement Learning (ICRL)
  11. Adaptive Curriculum Learning
  12. (a) Relevance-Driven Skill Grouping.
  13. (b) Helpfulness-Driven Dynamic Curriculum.
  14. Experiment
  15. Experiment Setup
  16. ...and 20 more sections

Figures (12)

  • Figure 1: Comparison of (a) Skill Augmentation methods and (b) our Skill Internalization method.
  • Figure 2: Overview of Skill0. (a) Relevance-Driven Skill Grouping; (b) In-Context Reinforcement Learning with skill-enhanced agent loop; (c) Dynamic curriculum learning during training process.
  • Figure 3: Comparison of training dynamics with AgentOCR on Qwen2.5-VL-3B.
  • Figure 4: Comparison of training dynamics with AgentOCR on Qwen2.5-VL-7B.
  • Figure 5: Training Dynamics Comparison. (a) Validation performance of Skill0 (OCR) with and without skill augmentation, evaluated every 10 training steps. (b) Performance comparison between Skill0 and AgentOCR, both evaluated without skill augmentation. (c) Performance comparison of Skill0 (OCR) against GRPO (Text) and SkillRL (Text), all evaluated without skill augmentation.
  • ...and 7 more figures