MACIE: Multi-Agent Causal Intelligence Explainer for Collective Behavior Understanding

TL;DR

MACIE presents a principled framework for explainable collective intelligence in multi-agent RL by unifying structural causal models, interventional counterfactuals, and Shapley-value attribution. It introduces novel collective intelligence metrics—Synergy Index, Coordination Score, and Information Integration—to quantify emergent behavior and distinguish it from individual contributions, while generating natural language explanations for stakeholders. The approach is implemented with computational optimizations that deliver sub-second explanations on CPU and is validated across four MARL scenarios, achieving accurate attributions (mean |φ_i| ≈ 5.07) and robust emergence detection (SI up to 0.461). This work advances trustworthy MARL by enabling fair credit assignment, emergent-behavior analysis, and actionable explanations with practical efficiency for real-world deployment.

Abstract

As Multi Agent Reinforcement Learning systems are used in safety critical applications. Understanding why agents make decisions and how they achieve collective behavior is crucial. Existing explainable AI methods struggle in multi agent settings. They fail to attribute collective outcomes to individuals, quantify emergent behaviors, or capture complex interactions. We present MACIE Multi Agent Causal Intelligence Explainer, a framework combining structural causal models, interventional counterfactuals, and Shapley values to provide comprehensive explanations. MACIE addresses three questions. First, each agent's causal contribution using interventional attribution scores. Second, system level emergent intelligence through synergy metrics separating collective effects from individual contributions. Third, actionable explanations using natural language narratives synthesizing causal insights. We evaluate MACIE across four MARL scenarios: cooperative, competitive, and mixed motive. Results show accurate outcome attribution, mean phi_i equals 5.07, standard deviation less than 0.05, detection of positive emergence in cooperative tasks, synergy index up to 0.461, and efficient computation, 0.79 seconds per dataset on CPU. MACIE uniquely combines causal rigor, emergence quantification, and multi agent support while remaining practical for real time use. This represents a step toward interpretable, trustworthy, and accountable multi agent AI.

Figures (4)

• Figure 1: Causal attribution scores ($\phi_i$) across all datasets. Green bars indicate positive contributions ($\phi_i > 0$), red bars indicate negative contributions ($\phi_i < 0$). Synergy Index (SI) values are annotated in each subplot. Positive SI indicates emergent synergy; negative SI indicates interference or competition.
• Figure 2: Box plots comparing factual outcomes (leftmost, green) with counterfactual outcomes where each agent is replaced by random baseline (red). Larger differences between factual and counterfactual distributions indicate higher causal contributions. Whiskers show min/max values; boxes show quartiles.
• Figure 3: Runtime breakdown by algorithm step, averaged across all datasets. Counterfactual generation (Step 2) dominates due to multiple episode simulations ($K \times N$ episodes). Shapley computation (Step 4) scales with coalition samples $M$. Causal model learning (Step 1) remains efficient with Random Forest.
• Figure 4: Convergence of attribution scores $\phi_i$ as a function of counterfactual samples $K$. Shaded regions show 95% confidence intervals from bootstrap resampling. Scores stabilize at $K \geq 5$, justifying our choice of $K=5$ for computational efficiency.

Theorems & Definitions (13)

• Definition 3.1: Markov Decision Process
• Definition 3.2: Policy
• Definition 3.3: Value Functions
• Definition 3.4: Stochastic Game
• Definition 3.5: Joint Policy
• Definition 3.6: Dec-POMDP
• Definition 3.7: Nash Equilibrium
• Definition 3.8: Structural Causal Model
• Definition 3.9: Intervention
• Definition 3.10: Counterfactual