Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RE-GAINS & EnChAnT: Intelligent Tool Manipulation Systems For Enhanced Query Responses

Sahil Girhepuje, Siva Sankar Sajeev, Purvam Jain, Arya Sikder, Adithya Rama Varma, Ryan George, Akshay Govind Srinivasan, Mahendra Kurup, Ashmit Sinha, Sudip Mondal

TL;DR

This work introduces two tool-augmented LLM pipelines, RE-GAINS and EnChAnT, designed to enable intelligent API tool invocation and multi-tool chaining with minimal latency and cost. RE-GAINS relies on retrieval-augmented planning (RAP) to maintain a world-model of states and actions, while EnChAnT emphasizes an LM Format Enforcer and open-source components to suppress hallucinations and ensure robust outputs. The authors provide extensive experimentation, including prompting strategies, graph-based type checking, reflexive reasoning loops, and enforcers, and demonstrate cost-effective performance (approximately $0.01 per query) against baselines like ControlLLM and RAP with GPT-4. They also detail data-generation pipelines (multi-agent and persona-based) and tool-retrieval architectures leveraging Ada embeddings and ToolBench retrievers. The work highlights promising directions for scalable, domain-adaptive tool use in LLMs and outlines concrete future work on reasoning trees and domain classification.

Abstract

Large Language Models (LLMs) currently struggle with tool invocation and chaining, as they often hallucinate or miss essential steps in a sequence. We propose RE-GAINS and EnChAnT, two novel frameworks that empower LLMs to tackle complex user queries by making API calls to external tools based on tool descriptions and argument lists. Tools are chained based on the expected output, without receiving the actual results from each individual call. EnChAnT, an open-source solution, leverages an LLM format enforcer, OpenChat 3.5 (an LLM), and ToolBench's API Retriever. RE-GAINS utilizes OpenAI models and embeddings with a specialized prompt based on the $\underline{R}$easoning vi$\underline{a}$ $\underline{P}$lanning $(RAP)$ framework. Both frameworks are low cost (0.01\$ per query). Our key contribution is enabling LLMs for tool invocation and chaining using modifiable, externally described tools.

RE-GAINS & EnChAnT: Intelligent Tool Manipulation Systems For Enhanced Query Responses

TL;DR

This work introduces two tool-augmented LLM pipelines, RE-GAINS and EnChAnT, designed to enable intelligent API tool invocation and multi-tool chaining with minimal latency and cost. RE-GAINS relies on retrieval-augmented planning (RAP) to maintain a world-model of states and actions, while EnChAnT emphasizes an LM Format Enforcer and open-source components to suppress hallucinations and ensure robust outputs. The authors provide extensive experimentation, including prompting strategies, graph-based type checking, reflexive reasoning loops, and enforcers, and demonstrate cost-effective performance (approximately $0.01 per query) against baselines like ControlLLM and RAP with GPT-4. They also detail data-generation pipelines (multi-agent and persona-based) and tool-retrieval architectures leveraging Ada embeddings and ToolBench retrievers. The work highlights promising directions for scalable, domain-adaptive tool use in LLMs and outlines concrete future work on reasoning trees and domain classification.

Abstract

Large Language Models (LLMs) currently struggle with tool invocation and chaining, as they often hallucinate or miss essential steps in a sequence. We propose RE-GAINS and EnChAnT, two novel frameworks that empower LLMs to tackle complex user queries by making API calls to external tools based on tool descriptions and argument lists. Tools are chained based on the expected output, without receiving the actual results from each individual call. EnChAnT, an open-source solution, leverages an LLM format enforcer, OpenChat 3.5 (an LLM), and ToolBench's API Retriever. RE-GAINS utilizes OpenAI models and embeddings with a specialized prompt based on the easoning vi lanning framework. Both frameworks are low cost (0.01\$ per query). Our key contribution is enabling LLMs for tool invocation and chaining using modifiable, externally described tools.
Paper Structure (65 sections, 10 figures, 10 tables)

This paper contains 65 sections, 10 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Prompting Techniques
  4. Existing literature on Tool-based LLMs
  5. Benchmarks
  6. Limited Tools Usage
  7. Embeddings for Tool Retrieval
  8. Agent-Based learning
  9. Discussion on Latency and Cost
  10. Discussion on Hallucination
  11. Ideas on Data
  12. Data Generation
  13. Multi-Agent Framework
  14. Conversation Framework
  15. Experiment A: Flow Based Generation
  16. ...and 50 more sections

Figures (10)

  • Figure 1: A novel pipeline to use LLM enforcers on closed-sourced models like GPT 3.5
  • Figure 2: Pipeline for the main proposed solution. The dotted lines represent the optional component of the process.
  • Figure 3: Pipeline of Proposed Architecture
  • Figure 4: GitHub Copilot auto-completing the entire user query
  • Figure 5: An example of a tool generated by us
  • ...and 5 more figures