GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent Collaboration

TL;DR

GraphTeam addresses the challenge of scalable graph reasoning with LLMs by introducing a structured multi-agent system that coordinates specialized modules for input understanding, knowledge retrieval, and problem solving. The five agents operate in a division-of-labor workflow, with a question agent, search agent, coding and reasoning agents, and an answer agent, supported by a retrieval-augmented knowledge base built from documentation and past experiences. The knowledge base feeds a retrieval mechanism via vector databases (LlamaIndex) and a thresholded similarity check to supply relevant information for coding or reasoning, while a retry mechanism keeps improving code-based solutions. Across six graph reasoning benchmarks, GraphTeam achieves state-of-the-art accuracy improvements and ablation studies demonstrate the necessity of each component, underscoring the practical value of external knowledge and collaboration for graph analysis.

Abstract

Graphs are widely used for modeling relational data in real-world scenarios, such as social networks and urban computing. Existing LLM-based graph analysis approaches either integrate graph neural networks (GNNs) for specific machine learning tasks, limiting their transferability, or rely solely on LLMs' internal reasoning ability, resulting in suboptimal performance. To address these limitations, we take advantage of recent advances in LLM-based agents, which have shown capabilities of utilizing external knowledge or tools for problem solving. By simulating human problem-solving strategies such as analogy and collaboration, we propose a multi-agent system based on LLMs named GraphTeam, for graph analysis. GraphTeam consists of five LLM-based agents from three modules, and the agents with different specialities can collaborate with each other to address complex problems. Specifically, (1) input-output normalization module: the question agent extracts and refines four key arguments from the original question, facilitating the problem understanding, and the answer agent organizes the results to meet the output requirement; (2) external knowledge retrieval module: we first build a knowledge base consisting of relevant documentation and experience information, and then the search agent retrieves the most relevant entries for each question. (3) problem-solving module: given the retrieved information from search agent, the coding agent uses established algorithms via programming to generate solutions, and in case the coding agent does not work, the reasoning agent will directly compute the results without programming. Extensive experiments on six graph analysis benchmarks demonstrate that GraphTeam achieves state-of-the-art performance with an average 25.85% improvement over the best baseline in terms of accuracy. The code and data are available at https://github.com/BUPT-GAMMA/GraphTeam.

Figures (9)

• Figure 1: The overall pipeline of our multi-agent system GraphTeam (left), and the comparison between GraphTeam and the best baseline on six benchmarks (right).
• Figure 2: The overall framework of GraphTeam, which includes five agents across three functional groups. Firstly, the question agent refines the original question by extracting key arguments. Then the search agent queries knowledge bases with the reformatted question to retrieve relevant experience and documentation. Subsequently, the coding agent attempts to solve the problem via programming. If the coding agent fails after several retries, the reasoning agent will take over and directly solve the question without programming. Finally, the solutions from either the coding or reasoning agents are then passed to the answer agent for format standardization.
• Figure 3: Performance with respect to different task categories.
• Figure 4: Performance with respect to different output formats.
• Figure 5: Average accuracies (%) and token consumptions of different agent-based methods. While maintaining comparable token consumption with typical agent frameworks, GraphTeam substantially improves the average accuracy for graph reasoning.