Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectories

Yifan Song, Weimin Xiong, Xiutian Zhao, Dawei Zhu, Wenhao Wu, Ke Wang, Cheng Li, Wei Peng, Sujian Li

TL;DR

This work introduces AgentBank, by far the largest trajectory tuning data collection featuring more than 50k diverse high-quality interaction trajectories which comprises 16 tasks covering five distinct agent skill dimensions and fine-tune LLMs on AgentBank to get a series of agent models, Samoyed.

Abstract

Fine-tuning on agent-environment interaction trajectory data holds significant promise for surfacing generalized agent capabilities in open-source large language models (LLMs). In this work, we introduce AgentBank, by far the largest trajectory tuning data collection featuring more than 50k diverse high-quality interaction trajectories which comprises 16 tasks covering five distinct agent skill dimensions. Leveraging a novel annotation pipeline, we are able to scale the annotated trajectories and generate a trajectory dataset with minimized difficulty bias. Furthermore, we fine-tune LLMs on AgentBank to get a series of agent models, Samoyed. Our comparative experiments demonstrate the effectiveness of scaling the interaction trajectory data to acquire generalized agent capabilities. Additional studies also reveal some key observations regarding trajectory tuning and agent skill generalization.

AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectories

TL;DR

This work introduces AgentBank, by far the largest trajectory tuning data collection featuring more than 50k diverse high-quality interaction trajectories which comprises 16 tasks covering five distinct agent skill dimensions and fine-tune LLMs on AgentBank to get a series of agent models, Samoyed.

Abstract

Fine-tuning on agent-environment interaction trajectory data holds significant promise for surfacing generalized agent capabilities in open-source large language models (LLMs). In this work, we introduce AgentBank, by far the largest trajectory tuning data collection featuring more than 50k diverse high-quality interaction trajectories which comprises 16 tasks covering five distinct agent skill dimensions. Leveraging a novel annotation pipeline, we are able to scale the annotated trajectories and generate a trajectory dataset with minimized difficulty bias. Furthermore, we fine-tune LLMs on AgentBank to get a series of agent models, Samoyed. Our comparative experiments demonstrate the effectiveness of scaling the interaction trajectory data to acquire generalized agent capabilities. Additional studies also reveal some key observations regarding trajectory tuning and agent skill generalization.

Paper Structure

This paper contains 48 sections, 1 equation, 5 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Instruction Tuning
  4. LLM-based Agent
  5. Preliminary
  6. Agent Task Formulation
  7. Challenges in Trajectory Collection
  8. Hard to Scale-Up
  9. Difficulty Bias
  10. AgentBank
  11. Task and Instruction Collection
  12. Action Annotation
  13. Answer Forcing
  14. Heuristic Action Search
  15. Reformat
  16. ...and 33 more sections

Figures (5)

  • Figure 1: Overview of the construction process of AgentBank and the training procedure of Samoyed
  • Figure 2: The results of different base models. "Base" denotes untrained LLMs. "+SuperAgent" denotes models after training on AgentBank.
  • Figure 3: Scaling trends of the number of tasks and interaction trajectories.
  • Figure 4: Ablation study on data mixture.
  • Figure 5: Heatmap of skill-level capability transfer. We plot the relative improvements over training on generalist instruction and code data.