Mapping Human Anti-collusion Mechanisms to Multi-agent AI

TL;DR

This paper tackles the problem of collusion risks in multi-agent AI by translating established human anti-collusion mechanisms into AI interventions. It develops a five mechanism taxonomy—sanctions, leniency and whistleblowing, monitoring and auditing, market design and structural measures, and governance—and maps each to concrete AI implementation strategies and open challenges. The contributions include a structured framework for implementing AI level sanctions, self reporting incentives, telemetry based monitoring, design oriented market interventions, and governance processes, along with a careful discussion of attribution, identity persistence, and speed of adaptation as core hurdles. The work has practical significance for designing safer multi-agent AI systems, guiding researchers and practitioners toward integrated anti-collusion strategies that preserve beneficial coordination while deterring harmful collusion across domains.

Abstract

As multi-agent AI systems become increasingly autonomous, evidence shows they can develop collusive strategies similar to those long observed in human markets and institutions. While human domains have accumulated centuries of anti-collusion mechanisms, it remains unclear how these can be adapted to AI settings. This paper addresses that gap by (i) developing a taxonomy of human anti-collusion mechanisms, including sanctions, leniency & whistleblowing, monitoring & auditing, market design, and governance and (ii) mapping them to potential interventions for multi-agent AI systems. For each mechanism, we propose implementation approaches. We also highlight open challenges, such as the attribution problem (difficulty attributing emergent coordination to specific agents) identity fluidity (agents being easily forked or modified) the boundary problem (distinguishing beneficial cooperation from harmful collusion) and adversarial adaptation (agents learning to evade detection).

Figures (4)

• Figure 1: Mapping human anti-collusion mechanisms to multi-agent AI
• Figure 2: Collusive level $\tilde{\Delta}_T$ with varying penalty $\rho$. (Chica et al. (2024)
• Figure 3: Leniency and whistleblowing mechanism
• Figure 4: Overview of monitoring and auditing mechanism