Cross-Task Experiential Learning on LLM-based Multi-Agent Collaboration

TL;DR

MAEL addresses cross-task generalization in LLM-based multi-agent systems by accumulating task experiences in per-agent pools during an experiential learning phase and retrieving those experiences during inference to guide reasoning steps. The framework models the MAS as a graph with per-agent pools and uses a forward–backward pass mechanism to annotate and store stepwise rewards, guiding future task solving. Across GSM-8K, MMLU, HumanEval, CommonGen-Hard, and SRDD, MAEL variants achieve faster convergence and higher-quality solutions than strong baselines, with step-wise retrieval providing the largest gains. This cross-task transfer improves both efficiency and robustness of multi-agent collaboration, suggesting practical benefits for complex, domain-spanning problem solving.

Abstract

Large Language Model-based multi-agent systems (MAS) have shown remarkable progress in solving complex tasks through collaborative reasoning and inter-agent critique. However, existing approaches typically treat each task in isolation, resulting in redundant computations and limited generalization across structurally similar tasks. To address this, we introduce multi-agent cross-task experiential learning (MAEL), a novel framework that endows LLM-driven agents with explicit cross-task learning and experience accumulation. We model the task-solving workflow on a graph-structured multi-agent collaboration network, where agents propagate information and coordinate via explicit connectivity. During the experiential learning phase, we quantify the quality for each step in the task-solving workflow and store the resulting rewards along with the corresponding inputs and outputs into each agent's individual experience pool. During inference, agents retrieve high-reward, task-relevant experiences as few-shot examples to enhance the effectiveness of each reasoning step, thereby enabling more accurate and efficient multi-agent collaboration. Experimental results on diverse datasets demonstrate that MAEL empowers agents to learn from prior task experiences effectively-achieving faster convergence and producing higher-quality solutions on current tasks.

Figures (5)

• Figure 1: MAEL casts MAS as a graph of LLM-based agents, each endowed with an experience pool acting as learnable weights. During the experiential learning phase, agents execute tasks via a divide-and-conquer plus critique forward pass, collect state–action pairs annotated with rewards at every key decision step, and then perform a backpropagation-style update by storing quantified experiences into their own experience pools. In the inference phase, incoming tasks are solved through the same task-solving workflow, but each agent augments its reasoning with retrieved experiences—selecting high-quality task-relevant past experiences based on a combined similarity-and-reward score—as few-shot examples, guiding subtask decomposition, solution, refinement, and aggregation.
• Figure 2: Comparison of token consumption (Left) and number of convergence round (Right). Since the SRDD dataset requires significantly more rounds and tokens than other datasets, for better visualization, we normalize all values relative to MAELØExp.
• Figure 3: Performance of MAELTask on HumanEval and CommonGen benchmarks with varying experience pool sizes (0, 10, 20, and 30 accumulated tasks).
• Figure 4: Solution quality comparison across five cases using different experience retrieval methods.
• Figure 5: A case illustrating the assistance of experience (yellow).