Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Advancing Agentic Systems: Dynamic Task Decomposition, Tool Integration and Evaluation using Novel Metrics and Dataset

Adrian Garret Gabriel, Alaa Alameer Ahmad, Shankar Kumar Jeyakumar

TL;DR

The findings reveal that asynchronous and dynamic task graph decomposition significantly enhances system responsiveness and scalability, particularly for complex, multi-step tasks.

Abstract

Advancements in Large Language Models (LLMs) are revolutionizing the development of autonomous agentic systems by enabling dynamic, context-aware task decomposition and automated tool selection. These sophisticated systems possess significant automation potential across various industries, managing complex tasks, interacting with external systems to enhance knowledge, and executing actions independently. This paper presents three primary contributions to advance this field: - Advanced Agentic Framework: A system that handles multi-hop queries, generates and executes task graphs, selects appropriate tools, and adapts to real-time changes. - Novel Evaluation Metrics: Introduction of Node F1 Score, Structural Similarity Index (SSI), and Tool F1 Score to comprehensively assess agentic systems. - Specialized Dataset: Development of an AsyncHow-based dataset for analyzing agent behavior across different task complexities. Our findings reveal that asynchronous and dynamic task graph decomposition significantly enhances system responsiveness and scalability, particularly for complex, multi-step tasks. Detailed analysis shows that structural and node-level metrics are crucial for sequential tasks, while tool-related metrics are more important for parallel tasks. Specifically, the Structural Similarity Index (SSI) is the most significant predictor of performance in sequential tasks, and the Tool F1 Score is essential for parallel tasks. These insights highlight the need for balanced evaluation methods that capture both structural and operational dimensions of agentic systems. Additionally, our evaluation framework, validated through empirical analysis and statistical testing, provides valuable insights for improving the adaptability and reliability of agentic systems in dynamic environments.

Advancing Agentic Systems: Dynamic Task Decomposition, Tool Integration and Evaluation using Novel Metrics and Dataset

TL;DR

The findings reveal that asynchronous and dynamic task graph decomposition significantly enhances system responsiveness and scalability, particularly for complex, multi-step tasks.

Abstract

Advancements in Large Language Models (LLMs) are revolutionizing the development of autonomous agentic systems by enabling dynamic, context-aware task decomposition and automated tool selection. These sophisticated systems possess significant automation potential across various industries, managing complex tasks, interacting with external systems to enhance knowledge, and executing actions independently. This paper presents three primary contributions to advance this field: - Advanced Agentic Framework: A system that handles multi-hop queries, generates and executes task graphs, selects appropriate tools, and adapts to real-time changes. - Novel Evaluation Metrics: Introduction of Node F1 Score, Structural Similarity Index (SSI), and Tool F1 Score to comprehensively assess agentic systems. - Specialized Dataset: Development of an AsyncHow-based dataset for analyzing agent behavior across different task complexities. Our findings reveal that asynchronous and dynamic task graph decomposition significantly enhances system responsiveness and scalability, particularly for complex, multi-step tasks. Detailed analysis shows that structural and node-level metrics are crucial for sequential tasks, while tool-related metrics are more important for parallel tasks. Specifically, the Structural Similarity Index (SSI) is the most significant predictor of performance in sequential tasks, and the Tool F1 Score is essential for parallel tasks. These insights highlight the need for balanced evaluation methods that capture both structural and operational dimensions of agentic systems. Additionally, our evaluation framework, validated through empirical analysis and statistical testing, provides valuable insights for improving the adaptability and reliability of agentic systems in dynamic environments.

Paper Structure

This paper contains 25 sections, 4 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Agentic Frameworks for Task Graph Generation and Tool Selection
  4. Evaluation Metrics and Datasets for Agentic Frameworks
  5. Agentic Framework Architecture
  6. Evaluation Framework
  7. Dataset Creation
  8. Evaluation Metrics
  9. Results and Discussion
  10. Limitations
  11. Future Work
  12. Conclusion
  13. Replicating the Agentic Task Decomposition Framework
  14. Prerequisites
  15. Core Components and Function Shells
  16. ...and 10 more sections

Figures (5)

  • Figure 1: Overview of the framework architecture, showcasing the flow of a user query through the system.
  • Figure 2: Execution flow managed by the Executor showcasing a simpler task flow on the left and a more complex flow on the right.
  • Figure 3: Feature Correlations with Answer Score for Sequential Tasks
  • Figure 4: Feature Correlations with Answer Score for Parallel Tasks
  • Figure 5: Task execution timings for parallel execution. The diagram shows the start and end times for four tasks (Task 1, Task 2, Task 3, and Task 4). Tasks 1 and 4, which are independent, are executed in parallel, while Task 2 starts after Task 4, and Task 3 follows Task 2, showing sequential execution due to dependencies.