Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Text Classification with a Novel Multi-Agent Collaboration Framework Leveraging BERT

Hediyeh Baban, Sai A Pidapar, Aashutosh Nema, Sichen Lu

TL;DR

This work introduces a threshold-driven, multi-agent collaboration framework that extends a BERT-based text classifier with five specialized agents (Lexical, Contextual, Logic, Consensus, Explainability) to handle low-confidence predictions. By dynamically escalating uncertain cases and aggregating agent insights through a Consensus mechanism, the approach achieves up to a 5.5% accuracy improvement and notable robustness gains across diverse NLP tasks, while detailing production-oriented considerations such as modularity, efficiency, and security. The methodology is grounded in ensemble and reinforcement-inspired reasoning, with explicit mathematical formulations for agent collaboration and decision aggregation. Empirical results on standard datasets (e.g., IMDb, AG News, SMS Spam, and a custom Intent dataset) demonstrate significant performance benefits over standard BERT, BERT ensembles, and CAMEL, suggesting practical impact for high-stakes text classification scenarios.

Abstract

We introduce a novel multi-agent collaboration framework designed to enhance the accuracy and robustness of text classification models. Leveraging BERT as the primary classifier, our framework dynamically escalates low-confidence predictions to a specialized multi-agent system comprising Lexical, Contextual, Logic, Consensus, and Explainability agents. This collaborative approach allows for comprehensive analysis and consensus-driven decision-making, significantly improving classification performance across diverse text classification tasks. Empirical evaluations on benchmark datasets demonstrate that our framework achieves a 5.5% increase in accuracy compared to standard BERT-based classifiers, underscoring its effectiveness and academic novelty in advancing multi-agent systems within natural language processing.

Enhancing Text Classification with a Novel Multi-Agent Collaboration Framework Leveraging BERT

TL;DR

This work introduces a threshold-driven, multi-agent collaboration framework that extends a BERT-based text classifier with five specialized agents (Lexical, Contextual, Logic, Consensus, Explainability) to handle low-confidence predictions. By dynamically escalating uncertain cases and aggregating agent insights through a Consensus mechanism, the approach achieves up to a 5.5% accuracy improvement and notable robustness gains across diverse NLP tasks, while detailing production-oriented considerations such as modularity, efficiency, and security. The methodology is grounded in ensemble and reinforcement-inspired reasoning, with explicit mathematical formulations for agent collaboration and decision aggregation. Empirical results on standard datasets (e.g., IMDb, AG News, SMS Spam, and a custom Intent dataset) demonstrate significant performance benefits over standard BERT, BERT ensembles, and CAMEL, suggesting practical impact for high-stakes text classification scenarios.

Abstract

We introduce a novel multi-agent collaboration framework designed to enhance the accuracy and robustness of text classification models. Leveraging BERT as the primary classifier, our framework dynamically escalates low-confidence predictions to a specialized multi-agent system comprising Lexical, Contextual, Logic, Consensus, and Explainability agents. This collaborative approach allows for comprehensive analysis and consensus-driven decision-making, significantly improving classification performance across diverse text classification tasks. Empirical evaluations on benchmark datasets demonstrate that our framework achieves a 5.5% increase in accuracy compared to standard BERT-based classifiers, underscoring its effectiveness and academic novelty in advancing multi-agent systems within natural language processing.

Paper Structure

This paper contains 41 sections, 7 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Workflow Overview
  5. Primary Classification with BERT
  6. Threshold-Based Escalation
  7. Multi-Agent Collaboration
  8. Agent Interaction Protocol
  9. Example Agent Conversation
  10. Detailed Agent Conversation
  11. Mathematical Formulation of Agent Collaboration
  12. Mathematical Justification for Improvement
  13. Experiments
  14. Datasets
  15. Baselines
  16. ...and 26 more sections

Figures (3)

  • Figure 1: Architecture of the Multi-Agent Collaboration Framework for Text Classification. The primary BERT classifier processes input text and assigns a label with a confidence score. If the confidence score is below the threshold $\tau$, the input is escalated to the multi-agent system comprising Lexical, Contextual, Logic, Consensus, and Explainability agents.
  • Figure 2: Agent Interaction Protocol within the Multi-Agent Collaboration System. Each agent communicates with others to refine the classification decision.
  • Figure 3: Precision-Recall Curve comparing BERT and BERT + Multi-Agent on the IMDB Dataset.