Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learning

Xinzhe Li

TL;DR

The paper addresses the challenge of comparing LLM-based agents across tool use, planning, and feedback learning by presenting a unified taxonomy built on universal LLM-profiled roles (LMPRs). It analyzes task universality across decision-making and NLIE environments, and compares a range of base, tool-use, search, and feedback-learning workflows through the lens of LMPRs. Key contributions include a framework for evaluating cross-paradigm implementations, a discussion of prompting methods, and explicit limitations with directions for future cross-paradigm workflow design. The work aims to standardize understanding and accelerate development of versatile, task-agnostic LLM-based agents, with openly available resources on GitHub to support replication and extension.

Abstract

Tool use, planning, and feedback learning are currently three prominent paradigms for developing Large Language Model (LLM)-based agents across various tasks. Although numerous frameworks have been devised for each paradigm, their intricate workflows and inconsistent taxonomy create challenges in understanding and reviewing the frameworks across different paradigms. This survey introduces a unified taxonomy to systematically review and discuss these frameworks. Specifically, 1) the taxonomy defines environments/tasks, common LLM-profiled roles or LMPRs (policy models, evaluators, and dynamic models), and universally applicable workflows found in prior work, and 2) it enables a comparison of key perspectives on the implementations of LMPRs and workflow designs across different agent paradigms and frameworks. 3) Finally, we identify three limitations in existing workflow designs and systematically discuss the future work. Resources have been made publicly available at in our GitHub repository https://github.com/xinzhel/LLM-Agent-Survey.

A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learning

TL;DR

The paper addresses the challenge of comparing LLM-based agents across tool use, planning, and feedback learning by presenting a unified taxonomy built on universal LLM-profiled roles (LMPRs). It analyzes task universality across decision-making and NLIE environments, and compares a range of base, tool-use, search, and feedback-learning workflows through the lens of LMPRs. Key contributions include a framework for evaluating cross-paradigm implementations, a discussion of prompting methods, and explicit limitations with directions for future cross-paradigm workflow design. The work aims to standardize understanding and accelerate development of versatile, task-agnostic LLM-based agents, with openly available resources on GitHub to support replication and extension.

Abstract

Tool use, planning, and feedback learning are currently three prominent paradigms for developing Large Language Model (LLM)-based agents across various tasks. Although numerous frameworks have been devised for each paradigm, their intricate workflows and inconsistent taxonomy create challenges in understanding and reviewing the frameworks across different paradigms. This survey introduces a unified taxonomy to systematically review and discuss these frameworks. Specifically, 1) the taxonomy defines environments/tasks, common LLM-profiled roles or LMPRs (policy models, evaluators, and dynamic models), and universally applicable workflows found in prior work, and 2) it enables a comparison of key perspectives on the implementations of LMPRs and workflow designs across different agent paradigms and frameworks. 3) Finally, we identify three limitations in existing workflow designs and systematically discuss the future work. Resources have been made publicly available at in our GitHub repository https://github.com/xinzhel/LLM-Agent-Survey.
Paper Structure (41 sections, 1 equation, 1 figure, 15 tables)

This paper contains 41 sections, 1 equation, 1 figure, 15 tables.

Table of Contents

  1. Introduction
  2. Comparisons with Existing Surveys
  3. Contributions
  4. Task Universality
  5. Feedback-Based, Decision-Making Environments
  6. Rule-Based Game Environments
  7. Embodied Environments
  8. Information Processing Environments
  9. Web Environments
  10. Natural Language Interaction Environments
  11. LLM-Profiled Roles (LMPRs)
  12. LLM-Profiled Policy $\text{glm}_{\text{policy}}$
  13. LLM-Profiled Evaluator $\text{glm}_{\text{eval}}$
  14. LLM-Profiled Dynamic Models $\text{glm}_{\text{dynamic}}$
  15. LMPR-Based Workflows
  16. ...and 26 more sections

Figures (1)

  • Figure 1: Common workflows based on three LLM-Profiled Roles (LMPRs): Policy, Evaluator and Dynamic Model. Numbers in circles indicate the sequence of execution for each step. Unnumbered lines represent iterative steps. Tool-use steps autonomously determined by LMPRs are marked by dashed lines. Circles labeled "-1" in the circles indicate the final step. The 'Tool Invocation' and 'Search Tree/Graph' text boxes indicate that these processes/objects are maintained in agent programs.

Theorems & Definitions (3)

  • Remark 1
  • Remark 2
  • Remark 3