Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reverse Chain: A Generic-Rule for LLMs to Master Multi-API Planning

Yinger Zhang, Hui Cai, Xeirui Song, Yicheng Chen, Rui Sun, Jing Zheng

TL;DR

Reverse Chain reframes API planning as a backward, target-driven task for LLMs, decomposing multi-API calls into two manageable steps: API Selection and Argument Completion. By starting from the final API and iteratively filling arguments (including via internal API calls when needed), it reduces error propagation common in forward-planning methods and improves robustness across nesting levels. The authors created a 825-API compositional multi-tool dataset and an automatic GPT-4-based evaluator to benchmark in-context learning methods, showing that Reverse Chain outperforms CoT and ReAct, even in zero-shot settings. This approach offers strong practical implications for deploying LLMs as controllers of external tools with enhanced reliability and controllability, especially for complex, multi-tool tasks.

Abstract

While enabling large language models to implement function calling (known as APIs) can greatly enhance the performance of Large Language Models (LLMs), function calling is still a challenging task due to the complicated relations between different APIs, especially in a context-learning setting without fine-tuning. This paper introduces ``Reverse Chain'', a controllable, target-driven approach designed to empower LLMs with the capability to operate external APIs only via prompts. Recognizing that most LLMs have limited tool-use capabilities, Reverse Chain limits LLMs to executing simple tasks, e.g., API Selection and Argument Completion. Furthermore, to manage a controllable multi-function calling, Reverse Chain adopts a generic rule based on a backward reasoning process. This rule determines when to do API selection or Argument completion. To evaluate the multi-tool-use capability of LLMs, we have released a compositional multi-tool task dataset, available at \url{https://anonymous.4open.science/r/reverse-chain-8681}. Extensive numerical experiments validate the remarkable proficiency of Reverse Chain in managing multiple API calls.

Reverse Chain: A Generic-Rule for LLMs to Master Multi-API Planning

TL;DR

Reverse Chain reframes API planning as a backward, target-driven task for LLMs, decomposing multi-API calls into two manageable steps: API Selection and Argument Completion. By starting from the final API and iteratively filling arguments (including via internal API calls when needed), it reduces error propagation common in forward-planning methods and improves robustness across nesting levels. The authors created a 825-API compositional multi-tool dataset and an automatic GPT-4-based evaluator to benchmark in-context learning methods, showing that Reverse Chain outperforms CoT and ReAct, even in zero-shot settings. This approach offers strong practical implications for deploying LLMs as controllers of external tools with enhanced reliability and controllability, especially for complex, multi-tool tasks.

Abstract

While enabling large language models to implement function calling (known as APIs) can greatly enhance the performance of Large Language Models (LLMs), function calling is still a challenging task due to the complicated relations between different APIs, especially in a context-learning setting without fine-tuning. This paper introduces ``Reverse Chain'', a controllable, target-driven approach designed to empower LLMs with the capability to operate external APIs only via prompts. Recognizing that most LLMs have limited tool-use capabilities, Reverse Chain limits LLMs to executing simple tasks, e.g., API Selection and Argument Completion. Furthermore, to manage a controllable multi-function calling, Reverse Chain adopts a generic rule based on a backward reasoning process. This rule determines when to do API selection or Argument completion. To evaluate the multi-tool-use capability of LLMs, we have released a compositional multi-tool task dataset, available at \url{https://anonymous.4open.science/r/reverse-chain-8681}. Extensive numerical experiments validate the remarkable proficiency of Reverse Chain in managing multiple API calls.
Paper Structure (22 sections, 17 figures, 7 tables)

This paper contains 22 sections, 17 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reverse Chain: A Multi-API Planning Approach
  4. Reverse Chaining
  5. LLM Modules in Reverse Chain
  6. API Selection
  7. Argument Completion
  8. Experiments
  9. Main Results
  10. Ablation Study
  11. Creativity and imagination of LLMs on Reverse Chain
  12. Argument Completion Optimization
  13. Reverse Chain_one-by-one
  14. Reverse Chain_three-step
  15. Why Reverse Chain works?
  16. ...and 7 more sections

Figures (17)

  • Figure 1: A comparison of our Reverse Chain with the one-step/CoT Planning and ReAct for multi-API planning.
  • Figure 2: Workflow of Reverse Chain on an example.
  • Figure 3: The details of prompts used in Reverse Chain for API Selection and Argument Completion (when LLM is chatgpt).
  • Figure 4: An example of seed example.
  • Figure 5: An example of sample in dataset.
  • ...and 12 more figures