LLM Harmony: Multi-Agent Communication for Problem Solving

TL;DR

The paper introduces LLM Harmony, a multi-agent, persona-driven framework for autonomous problem solving that leverages role-specific chain-of-thought prompts inspired by CAMEL. Using a two-agent (expert/evaluator) setup with fixed parameters, it demonstrates improved arithmetic and commonsense reasoning without retraining, across GSM8K, SVAMP, and CSQA datasets. The results show notable accuracy gains over single-agent and simple multi-agent baselines, highlighting the potential of collaborative LLMs to tackle novel problems. Limitations include dataset-dependency and context-length constraints, with future work aimed at scaling, knowledge updating, and better information filtering to sustain performance gains.

Abstract

Large Language Models (LLMs) have revolutionized Natural Language Processing but exhibit limitations, particularly in autonomously addressing novel challenges such as reasoning and problem-solving. Traditional techniques like chain-of-thought prompting necessitate explicit human guidance. This paper introduces a novel multi-agent communication framework, inspired by the CAMEL model, to enhance LLMs' autonomous problem-solving capabilities. The framework employs multiple LLM agents, each with a distinct persona, engaged in role-playing communication, offering a nuanced and adaptable approach to diverse problem scenarios. Extensive experimentation demonstrates the framework's superior performance and adaptability, providing valuable insights into the collaborative potential of multiple agents in overcoming the limitations of individual models.