Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EcoAct: Economic Agent Determines When to Register What Action

Shaokun Zhang, Jieyu Zhang, Dujian Ding, Mirian Hipolito Garcia, Ankur Mallick, Daniel Madrigal, Menglin Xia, Victor Rühle, Qingyun Wu, Chi Wang

TL;DR

EcoAct is a tool using algorithm that allows LLMs to selectively register tools as needed, optimizing context use and reduces computational costs by over 50% in multiple steps reasoning tasks while maintaining performance, as demonstrated through extensive experiments.

Abstract

Recent advancements have enabled Large Language Models (LLMs) to function as agents that can perform actions using external tools. This requires registering, i.e., integrating tool information into the LLM context prior to taking actions. Current methods indiscriminately incorporate all candidate tools into the agent's context and retain them across multiple reasoning steps. This process remains opaque to LLM agents and is not integrated into their reasoning procedures, leading to inefficiencies due to increased context length from irrelevant tools. To address this, we introduce EcoAct, a tool using algorithm that allows LLMs to selectively register tools as needed, optimizing context use. By integrating the tool registration process into the reasoning procedure, EcoAct reduces computational costs by over 50% in multiple steps reasoning tasks while maintaining performance, as demonstrated through extensive experiments. Moreover, it can be plugged into any reasoning pipeline with only minor modifications to the prompt, making it applicable to LLM agents now and future.

EcoAct: Economic Agent Determines When to Register What Action

TL;DR

EcoAct is a tool using algorithm that allows LLMs to selectively register tools as needed, optimizing context use and reduces computational costs by over 50% in multiple steps reasoning tasks while maintaining performance, as demonstrated through extensive experiments.

Abstract

Recent advancements have enabled Large Language Models (LLMs) to function as agents that can perform actions using external tools. This requires registering, i.e., integrating tool information into the LLM context prior to taking actions. Current methods indiscriminately incorporate all candidate tools into the agent's context and retain them across multiple reasoning steps. This process remains opaque to LLM agents and is not integrated into their reasoning procedures, leading to inefficiencies due to increased context length from irrelevant tools. To address this, we introduce EcoAct, a tool using algorithm that allows LLMs to selectively register tools as needed, optimizing context use. By integrating the tool registration process into the reasoning procedure, EcoAct reduces computational costs by over 50% in multiple steps reasoning tasks while maintaining performance, as demonstrated through extensive experiments. Moreover, it can be plugged into any reasoning pipeline with only minor modifications to the prompt, making it applicable to LLM agents now and future.

Paper Structure

This paper contains 28 sections, 5 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Problem setup
  4. EcoAct
  5. Tool name as informer.
  6. tool_register as meta-tool.
  7. Experiments
  8. Experimental setup
  9. Main results
  10. More analysis
  11. Extension to complex reasoning strategy
  12. Skill-library evolution
  13. Ablations
  14. Single-Tool vs. multiple-tool registration
  15. Tool names vs. descriptions information in tool registration
  16. ...and 13 more sections

Figures (4)

  • Figure 1: Overview of EcoAct, illustrating its effects after being integrated into the single-trace reasoning algorithm ReAct, which can serve as a fundamental component of complex reasoning methods. (a) In ReAct, all tools are registered in advance, retaining full information of these tools within the LLM's operational context at each reasoning step. This leads to unnecessarily long contexts, as tools irrelevant to the current problem remain included. (b) In contrast, EcoAct leverages ReAct's intrinsic reasoning capabilities to register only the tools deemed necessary, based on their concise and distinct identifiers - tool names, thus addressing the mentioned efficiency issues.
  • Figure 2: (a) Average token costs required for tools at each decision step, compared with the actual token costs incurred by tools using the React Algorithm yao2022react, across six subsets of ToolBench qin2023toolllm. (b) Average token consumption percentages for each component of the tools in ToolBench qin2023toolllm.
  • Figure 3: The average token cost and pass rate performance across queries with different numbers of tools in various models. For analysis, queries are categorized into three tool scale levels: Level 1, Level 2, and Level 3, corresponding to tool counts of 0-10, 10-20, and more than 20, respectively. It is observed that EcoAct benefits significantly from using a large number of tools, achieving token savings of 54.35% and 53.82% in two models respectively, with large-scale tools (Level 3). Additionally, EcoAct also surpasses the baseline on queries with large-scale tools in pass rate.
  • Figure 4: Evolution of the ratio of registered tools to total available tools across reasoning steps for different models, highlighting the largest percentage tool reductions for two models within each subset. Notably, the final registered tools comprise approximately 30% of the total available tools across all subsets, indicating that EcoAct effectively mitigates excessive tool registrations.