Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models

Xuechen Liang, Yangfan He, Meiling Tao, Yinghui Xia, Jianhui Wang, Tianyu Shi, Jun Wang, JingSong Yang

TL;DR

The paper tackles the reliance of large language models on human-guided prompts and the computational burden of scaling. It introduces Collaborative Multi-Agent Tuning (CMAT), a framework that uses User, Assistant, and Checker roles with memory and environment feedback to train compact models (TinyAgent) via supervised fine-tuning, CoT/ReAct reasoning, and a feedback-driven policy update. Key contributions include the CMAT architecture, a memory-management and self-reflection mechanism, and demonstrated parity or near-parity with larger models for TinyAgent-7B on several tasks, along with comprehensive ablation and error analyses. The work highlights the practical potential of efficient, multi-agent tuning to close the gap between small and large LLMs, enabling scalable deployment in resource-constrained settings.

Abstract

Open large language models (LLMs) have significantly advanced the field of natural language processing, showcasing impressive performance across various tasks.Despite the significant advancements in LLMs, their effective operation still relies heavily on human input to accurately guide the dialogue flow, with agent tuning being a crucial optimization technique that involves human adjustments to the model for better response to such guidance.Addressing this dependency, our work introduces the TinyAgent model, trained on a meticulously curated high-quality dataset. We also present the Collaborative Multi-Agent Tuning (CMAT) framework, an innovative system designed to augment language agent capabilities through adaptive weight updates based on environmental feedback. This framework fosters collaborative learning and real-time adaptation among multiple intelligent agents, enhancing their context-awareness and long-term memory. In this research, we propose a new communication agent framework that integrates multi-agent systems with environmental feedback mechanisms, offering a scalable method to explore cooperative behaviors. Notably, our TinyAgent-7B model exhibits performance on par with GPT-3.5, despite having fewer parameters, signifying a substantial improvement in the efficiency and effectiveness of LLMs.

CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models

TL;DR

The paper tackles the reliance of large language models on human-guided prompts and the computational burden of scaling. It introduces Collaborative Multi-Agent Tuning (CMAT), a framework that uses User, Assistant, and Checker roles with memory and environment feedback to train compact models (TinyAgent) via supervised fine-tuning, CoT/ReAct reasoning, and a feedback-driven policy update. Key contributions include the CMAT architecture, a memory-management and self-reflection mechanism, and demonstrated parity or near-parity with larger models for TinyAgent-7B on several tasks, along with comprehensive ablation and error analyses. The work highlights the practical potential of efficient, multi-agent tuning to close the gap between small and large LLMs, enabling scalable deployment in resource-constrained settings.

Abstract

Open large language models (LLMs) have significantly advanced the field of natural language processing, showcasing impressive performance across various tasks.Despite the significant advancements in LLMs, their effective operation still relies heavily on human input to accurately guide the dialogue flow, with agent tuning being a crucial optimization technique that involves human adjustments to the model for better response to such guidance.Addressing this dependency, our work introduces the TinyAgent model, trained on a meticulously curated high-quality dataset. We also present the Collaborative Multi-Agent Tuning (CMAT) framework, an innovative system designed to augment language agent capabilities through adaptive weight updates based on environmental feedback. This framework fosters collaborative learning and real-time adaptation among multiple intelligent agents, enhancing their context-awareness and long-term memory. In this research, we propose a new communication agent framework that integrates multi-agent systems with environmental feedback mechanisms, offering a scalable method to explore cooperative behaviors. Notably, our TinyAgent-7B model exhibits performance on par with GPT-3.5, despite having fewer parameters, signifying a substantial improvement in the efficiency and effectiveness of LLMs.
Paper Structure (20 sections, 8 equations, 3 figures, 5 tables)

This paper contains 20 sections, 8 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLMs Applications in a Multi-Agent Framework
  4. The tuning method for LLMs
  5. Methodology
  6. Agent Roles and Actor-Critic Dynamics
  7. Learning Strategy
  8. Supervised Fine-Tuning
  9. Incorporating CoT and ReAct
  10. Feedback-Driven Policy Optimization
  11. Checker-In-The-Loop Mechanism
  12. Memory Management and Self-Reflection
  13. Experiments
  14. Dataset
  15. Evaluating Code Correction
  16. ...and 5 more sections

Figures (3)

  • Figure 1: TinyAgent demonstrates outstanding performance, comparable to that of GPT-3.5. TinyAgent is a series of models fine-tuned based on Qwen bai2023qwen and Codellama roziere2023code.
  • Figure 2: In the CMAT framework, the user assigns tasks to an assistant, which generates SQL commands based on short-term and long-term memories: short-term memory provides immediate context from trajectory history, while self-reflective outputs are stored as long-term memory. The checker verifies the correctness of SQL commands before they are executed in the environment.
  • Figure 3: Comparative study of Llama-2-7b and TinyAgent-7b in DataBase cases. (1) In DataBase tasks with a reflection mechanism, Llama-2-7b still made errors after reflection, while TinyAgent-7b adjusted its operations after reflecting on its first failed attempt. (2) Without a reflection mechanism, TinyAgent-7b repeated the same operation and ultimately failed to complete the task.