Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Agent Actor-Critic Generative AI for Query Resolution and Analysis

Mohammad Wali Ur Rahman, Ric Nevarez, Lamia Tasnim Mim, Salim Hariri

TL;DR

MASQRAD presents a multi-agent actor-critic framework for translating vague natural language queries into precise data visualizations and analyses. The system leverages RoBERTa and LLaMA for query interpretation, GPT-3.5 Turbo/Codex for script generation, GPT-4-turbo for critique, and Claude-3.5 Sonnet plus GPT-4-omni for expert analyses, coordinated through a multi-agent debate mechanism to reduce hallucinations and improve reliability. Evaluated on nvBench and NL4DV with an 87% NL2VIS accuracy and demonstrated domain-agnostic performance (69.5% in a 200-query SEO setting), MASQRAD highlights the practical potential of robust, scalable AI-driven data interpretation while acknowledging limitations in dynamic schemas and domain generalization. The work contributes a concrete, scalable blueprint for integrating specialized AI agents to produce actionable visualizations and contextual reports, with significant implications for healthcare, finance, and policy decision support.

Abstract

In this paper, we introduce MASQRAD (Multi-Agent Strategic Query Resolution and Diagnostic tool), a transformative framework for query resolution based on the actor-critic model, which utilizes multiple generative AI agents. MASQRAD is excellent at translating imprecise or ambiguous user inquiries into precise and actionable requests. This framework generates pertinent visualizations and responses to these focused queries, as well as thorough analyses and insightful interpretations for users. MASQRAD addresses the common shortcomings of existing solutions in domains that demand fast and precise data interpretation, such as their incapacity to successfully apply AI for generating actionable insights and their challenges with the inherent ambiguity of user queries. MASQRAD functions as a sophisticated multi-agent system but "masquerades" to users as a single AI entity, which lowers errors and enhances data interaction. This approach makes use of three primary AI agents: Actor Generative AI, Critic Generative AI, and Expert Analysis Generative AI. Each is crucial for creating, enhancing, and evaluating data interactions. The Actor AI generates Python scripts to generate data visualizations from large datasets within operational constraints, and the Critic AI rigorously refines these scripts through multi-agent debate. Finally, the Expert Analysis AI contextualizes the outcomes to aid in decision-making. With an accuracy rate of 87\% when handling tasks related to natural language visualization, MASQRAD establishes new benchmarks for automated data interpretation and showcases a noteworthy advancement that has the potential to revolutionize AI-driven applications.

Multi-Agent Actor-Critic Generative AI for Query Resolution and Analysis

TL;DR

MASQRAD presents a multi-agent actor-critic framework for translating vague natural language queries into precise data visualizations and analyses. The system leverages RoBERTa and LLaMA for query interpretation, GPT-3.5 Turbo/Codex for script generation, GPT-4-turbo for critique, and Claude-3.5 Sonnet plus GPT-4-omni for expert analyses, coordinated through a multi-agent debate mechanism to reduce hallucinations and improve reliability. Evaluated on nvBench and NL4DV with an 87% NL2VIS accuracy and demonstrated domain-agnostic performance (69.5% in a 200-query SEO setting), MASQRAD highlights the practical potential of robust, scalable AI-driven data interpretation while acknowledging limitations in dynamic schemas and domain generalization. The work contributes a concrete, scalable blueprint for integrating specialized AI agents to produce actionable visualizations and contextual reports, with significant implications for healthcare, finance, and policy decision support.

Abstract

In this paper, we introduce MASQRAD (Multi-Agent Strategic Query Resolution and Diagnostic tool), a transformative framework for query resolution based on the actor-critic model, which utilizes multiple generative AI agents. MASQRAD is excellent at translating imprecise or ambiguous user inquiries into precise and actionable requests. This framework generates pertinent visualizations and responses to these focused queries, as well as thorough analyses and insightful interpretations for users. MASQRAD addresses the common shortcomings of existing solutions in domains that demand fast and precise data interpretation, such as their incapacity to successfully apply AI for generating actionable insights and their challenges with the inherent ambiguity of user queries. MASQRAD functions as a sophisticated multi-agent system but "masquerades" to users as a single AI entity, which lowers errors and enhances data interaction. This approach makes use of three primary AI agents: Actor Generative AI, Critic Generative AI, and Expert Analysis Generative AI. Each is crucial for creating, enhancing, and evaluating data interactions. The Actor AI generates Python scripts to generate data visualizations from large datasets within operational constraints, and the Critic AI rigorously refines these scripts through multi-agent debate. Finally, the Expert Analysis AI contextualizes the outcomes to aid in decision-making. With an accuracy rate of 87\% when handling tasks related to natural language visualization, MASQRAD establishes new benchmarks for automated data interpretation and showcases a noteworthy advancement that has the potential to revolutionize AI-driven applications.

Paper Structure

This paper contains 46 sections, 3 equations, 6 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Symbolic NLP Approaches
  4. Data-Driven NLP Approaches
  5. Multi-Agent AI Systems Overview
  6. Methodology
  7. Query Interpretation with RoBERTa and LLaMA
  8. Overview of RoBERTa
  9. Application of RoBERTa in Our System
  10. Overview of LLaMA
  11. Application of LLaMA in Our System
  12. Actor Agent-Driven Python Script Generation
  13. GPT-3.5 Turbo and Codex as Actor AI
  14. Mitigating Hallucination Error
  15. Functionality of the Actor Agent
  16. ...and 31 more sections

Figures (6)

  • Figure 1: Framework for Multi-Agent Strategic Query Resolution and Analytical Diagnostic Tool (MASQRAD)
  • Figure 2: Basic Architecture Diagram of a) RoBERTa, b) Llama and c) GPT
  • Figure 3: Performance Results of the trained interpreter RoBERTa models on the test query datasets for the (a) Movies Dataset (nvBench + NL4DV), (b) Cars Dataset (nvBench + NL4DV), (c) Superstore Dataset (NL4DV), (d) Inn Dataset (nvBench), (e) Euro Players Dataset (NL4DV) and (f) Architecture Dataset (nvBench)
  • Figure 4: Example of a Query Resolution by the MASQRAD System
  • Figure 5: Types of inaccuracies identified in the MASQRAD system during evaluation with the Vi(E)va LLM stack on subsets of nvBench and NL4DV benchmarks, categorized by evaluation layers.
  • ...and 1 more figures