Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SkillGen: Learning Domain Skills for In-Context Sequential Decision Making

Ruomeng Ding, Wei Cheng, Minglai Shao, Chen Zhao

TL;DR

SkillGen advances in-context learning for sequential decision making by learning domain-oriented, action-centric skills and using them to generate focused, step-aware prompts. It builds a domain knowledge graph from trajectories, assigns action utility with TD($\lambda$) credit, and extracts reusable skills centered on actions. A golden segment plus retrieved step-wise skills guide a frozen LLM to act without parameter updates, with theoretical results showing improved task identifiability from focused prompts. Empirically, SkillGen yields consistent gains across ALFWorld, BabyAI, and ScienceWorld on multiple models, and ablations confirm the contribution of golden segments, step-wise skills, and credit assignment. The approach also demonstrates strong token efficiency and adaptability to varying sampling and retrieval configurations, highlighting practical benefits for scalable, knowledge-augmented prompting in real-world robotic and reasoning tasks.

Abstract

Large language models (LLMs) are increasingly applied to sequential decision-making through in-context learning (ICL), yet their effectiveness is highly sensitive to prompt quality. Effective prompts should meet three principles: focus on decision-critical information, provide step-level granularity, and minimize reliance on expert annotations through label efficiency. However, existing ICL methods often fail to satisfy all three criteria simultaneously. Motivated by these challenges, we introduce SkillGen, a skill-based ICL framework for structured sequential reasoning. It constructs an action-centric, domain-level graph from sampled trajectories, identifies high-utility actions via temporal-difference credit assignment, and retrieves step-wise skills to generate fine-grained, context-aware prompts. We further present a theoretical analysis showing that focusing on high-utility segments supports task identifiability and informs more effective ICL prompt design. Experiments on ALFWorld, BabyAI, and ScienceWorld, using both open-source and proprietary LLMs, show that SkillGen achieves consistent gains, improving progress rate by 5.9%-16.5% on average across models.

SkillGen: Learning Domain Skills for In-Context Sequential Decision Making

TL;DR

SkillGen advances in-context learning for sequential decision making by learning domain-oriented, action-centric skills and using them to generate focused, step-aware prompts. It builds a domain knowledge graph from trajectories, assigns action utility with TD() credit, and extracts reusable skills centered on actions. A golden segment plus retrieved step-wise skills guide a frozen LLM to act without parameter updates, with theoretical results showing improved task identifiability from focused prompts. Empirically, SkillGen yields consistent gains across ALFWorld, BabyAI, and ScienceWorld on multiple models, and ablations confirm the contribution of golden segments, step-wise skills, and credit assignment. The approach also demonstrates strong token efficiency and adaptability to varying sampling and retrieval configurations, highlighting practical benefits for scalable, knowledge-augmented prompting in real-world robotic and reasoning tasks.

Abstract

Large language models (LLMs) are increasingly applied to sequential decision-making through in-context learning (ICL), yet their effectiveness is highly sensitive to prompt quality. Effective prompts should meet three principles: focus on decision-critical information, provide step-level granularity, and minimize reliance on expert annotations through label efficiency. However, existing ICL methods often fail to satisfy all three criteria simultaneously. Motivated by these challenges, we introduce SkillGen, a skill-based ICL framework for structured sequential reasoning. It constructs an action-centric, domain-level graph from sampled trajectories, identifies high-utility actions via temporal-difference credit assignment, and retrieves step-wise skills to generate fine-grained, context-aware prompts. We further present a theoretical analysis showing that focusing on high-utility segments supports task identifiability and informs more effective ICL prompt design. Experiments on ALFWorld, BabyAI, and ScienceWorld, using both open-source and proprietary LLMs, show that SkillGen achieves consistent gains, improving progress rate by 5.9%-16.5% on average across models.

Paper Structure

This paper contains 78 sections, 2 theorems, 33 equations, 7 figures, 21 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. In-Context Learning.
  4. LLMs for Sequential Decision-making.
  5. Method
  6. Domain Knowledge Construction
  7. Domain Skill Extraction
  8. TD-based credit assignment.
  9. Skill Extraction.
  10. Skill-Based In-Context Learning
  11. Theoretical Analysis
  12. Experimental Setup
  13. Datasets.
  14. Evaluation Metrics.
  15. Baselines.
  16. ...and 63 more sections

Key Result

Theorem 1

Let $p \sim P_{\phi^\star}^{\otimes k}$ be a prompt sampled from the true task $\phi^\star \in \Phi$, and suppose it admits a decomposition $p = p_{\text{focused}} \cup p_{\text{irrelevant}}$, where the partition is defined relative to $\phi^\star$. Suppose that $\min_{\phi \ne \phi^\star} \mathrm{K

Figures (7)

  • Figure 1: Framework of SkillGen.
  • Figure 2: Sparse $\mathcal{P}_{\Delta}(a_i, a_j)$ reward only subgoal completions, omitting intermediate actions.
  • Figure 3: Ablation study of SkillGen showing the effect of golden segments. Bars represent PR, $\Diamond$ markers indicate SR.
  • Figure 4: Ablation study of SkillGen showing the effect of step-wise skills. Bars represent PR, $\Diamond$ markers indicate SR.
  • Figure 5: Ablation study of SkillGen showing the effect of TD-based credit estimation. Bars represent PR, $\Diamond$ markers indicate SR.
  • ...and 2 more figures

Theorems & Definitions (3)

  • Theorem 1: Task Identifiability
  • Lemma 1: Task Likelihood Separation wies2023learnability
  • proof