Beyond Task Performance: A Metric-Based Analysis of Sequential Cooperation in Heterogeneous Multi-Agent Destructive Foraging

TL;DR

This work tackles cooperative behavior in heterogeneous multi-agent systems performing sequential destructive foraging under partial observability. It introduces a general, transferable suite of cooperation metrics organized into primary performance, inter-team, and intra-team categories and validates them in a dynamic, graph-based foraging environment with scouts (search) and foragers (destruction). The experimental results show that learning-based policies (DRL) yield faster early task progress and more balanced workload distributions, while heuristic approaches reveal distinct trade-offs, underscoring the value of the proposed metrics for diagnosing coordination dynamics beyond final task scores. The framework provides practical guidance for designing cooperative policies and can be extended to more complex team structures, energy constraints, and communication limitations, enhancing interpretability and robustness of multi-agent cooperation analyses.

Abstract

This work addresses the problem of analyzing cooperation in heterogeneous multi-agent systems which operate under partial observability and temporal role dependency, framed within a destructive multi-agent foraging setting. Unlike most previous studies, which focus primarily on algorithmic performance with respect to task completion, this article proposes a systematic set of general-purpose cooperation metrics aimed at characterizing not only efficiency, but also coordination and dependency between teams and agents, fairness, and sensitivity. These metrics are designed to be transferable to different multi-agent sequential domains similar to foraging. The proposed suite of metrics is structured into three main categories that jointly provide a multilevel characterization of cooperation: primary metrics, inter-team metrics, and intra-team metrics. They have been validated in a realistic destructive foraging scenario inspired by dynamic aquatic surface cleaning using heterogeneous autonomous vehicles. It involves two specialized teams with sequential dependencies: one focused on the search of resources, and another on their destruction. Several representative approaches have been evaluated, covering both learning-based algorithms and classical heuristic paradigms.

Figures (13)

• Figure 1: A team of agents with extended sensing capabilities explores the environment and discovers resources (scout), while a second team with limited sensing focuses on eliminating the discovered elements (forager).
• Figure 2: Scenario of the foraging environment with food distribution and differentiated field of vision of the scout and forager agents (left), discretized food distribution (middle), and model estimated based on the perceptions of the agents (right).
• Figure 3: Example of coverage overlap between two agents that share the same nodes in their field of view.
• Figure 4: Percentage of Target Achieved (PTA) metric for the objective of each team over normalized time, averaged across 100 episodes for 3 algorithms. Confidence intervals are represented as shaded regions.
• Figure 5: Root Mean Squared Error (RMSE) of the items distribution over normalized time, averaged across 100 episodes for 3 algorithm. Confidence intervals are represented as shaded regions.