Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TaskGen: A Task-Based, Memory-Infused Agentic Framework using StrictJSON

John Chong Min Tan, Prince Saroj, Bharat Runwal, Hardik Maheshwari, Brian Lim Yi Sheng, Richard Cottrill, Alankrit Chona, Ambuj Kumar, Mehul Motani

TL;DR

This work empirically evaluates TaskGen on various environments such as 40x40 dynamic maze navigation with changing obstacle locations, TextWorld escape room solving with dense rewards and detailed goals, web browsing, and Retrieval Augmented Generation on NaturalQuestions dataset.

Abstract

TaskGen is an open-sourced agentic framework which uses an Agent to solve an arbitrary task by breaking them down into subtasks. Each subtask is mapped to an Equipped Function or another Agent to execute. In order to reduce verbosity (and hence token usage), TaskGen uses StrictJSON that ensures JSON output from the Large Language Model (LLM), along with additional features such as type checking and iterative error correction. Key to the philosophy of TaskGen is the management of information/memory on a need-to-know basis. We empirically evaluate TaskGen on various environments such as 40x40 dynamic maze navigation with changing obstacle locations (100% solve rate), TextWorld escape room solving with dense rewards and detailed goals (96% solve rate), web browsing (69% of actions successful), solving the MATH dataset (71% solve rate over 100 Level-5 problems), Retrieval Augmented Generation on NaturalQuestions dataset (F1 score of 47.03%)

TaskGen: A Task-Based, Memory-Infused Agentic Framework using StrictJSON

TL;DR

This work empirically evaluates TaskGen on various environments such as 40x40 dynamic maze navigation with changing obstacle locations, TextWorld escape room solving with dense rewards and detailed goals, web browsing, and Retrieval Augmented Generation on NaturalQuestions dataset.

Abstract

TaskGen is an open-sourced agentic framework which uses an Agent to solve an arbitrary task by breaking them down into subtasks. Each subtask is mapped to an Equipped Function or another Agent to execute. In order to reduce verbosity (and hence token usage), TaskGen uses StrictJSON that ensures JSON output from the Large Language Model (LLM), along with additional features such as type checking and iterative error correction. Key to the philosophy of TaskGen is the management of information/memory on a need-to-know basis. We empirically evaluate TaskGen on various environments such as 40x40 dynamic maze navigation with changing obstacle locations (100% solve rate), TextWorld escape room solving with dense rewards and detailed goals (96% solve rate), web browsing (69% of actions successful), solving the MATH dataset (71% solve rate over 100 Level-5 problems), Retrieval Augmented Generation on NaturalQuestions dataset (F1 score of 47.03%)
Paper Structure (89 sections, 55 figures)

This paper contains 89 sections, 55 figures.

Table of Contents

  1. Introduction
  2. Motivation
  3. TaskGen Overall Design Philosophy
  4. The Core of TaskGen
  5. Agent Definition
  6. Imbuing Agentic Capabilities with Equipped Functions
  7. Choosing the Next Subtask
  8. Using TaskGen
  9. Benefits of TaskGen
  10. JSON is more concise than free text
  11. StrictJSON is more concise than JSON
  12. Modular and robust components
  13. Shared Memory
  14. Global Context
  15. Memory Bank
  16. ...and 74 more sections

Figures (55)

  • Figure 1: An Overview of TaskGen
  • Figure 2: Intractable action space when solving an arbitrary task
  • Figure 3: Constraining action space by Equipped Functions
  • Figure 4: Inner Agents assigned as Equipped Functions to a Meta Agent helps increase processing capability
  • Figure 5: More concise output using JSON as compared to Free Text using gpt-3.5-turbo on 12 Jul 2024
  • ...and 50 more figures