Adversarial Attack on Black-Box Multi-Agent by Adaptive Perturbation

TL;DR

AdapAM tackles the problem of assessing security in black-box MAS by learning to adaptively select a single victim agent $i_t$ and a malicious action $\tilde{a}_t$ to maximize impact while keeping perturbations under a budget $\epsilon$. It uses proxy agents trained with Multi-Agent Generative Adversarial Imitation Learning (MAGAIL) to approximate target policies, enabling white-box-like perturbation generation in a black-box setting. A CW-style perturbation generator crafts $\tilde{o}_{t,i}$ to satisfy $\pi_i(\tilde{o}_{t,i})=\tilde{a}_t$ with minimal $||o_{t,i}-\tilde{o}_{t,i}||$, balancing effectiveness and stealth. Across eight environments, AdapAM achieves superior attack performance and stealthiness against both normal and robust MAS, outperforming four strong baselines and highlighting MAS vulnerability to targeted, subtle perturbations under strict black-box constraints.

Abstract

Evaluating security and reliability for multi-agent systems (MAS) is urgent as they become increasingly prevalent in various applications. As an evaluation technique, existing adversarial attack frameworks face certain limitations, e.g., impracticality due to the requirement of white-box information or high control authority, and a lack of stealthiness or effectiveness as they often target all agents or specific fixed agents. To address these issues, we propose AdapAM, a novel framework for adversarial attacks on black-box MAS. AdapAM incorporates two key components: (1) Adaptive Selection Policy simultaneously selects the victim and determines the anticipated malicious action (the action would lead to the worst impact on MAS), balancing effectiveness and stealthiness. (2) Proxy-based Perturbation to Induce Malicious Action utilizes generative adversarial imitation learning to approximate the target MAS, allowing AdapAM to generate perturbed observations using white-box information and thus induce victims to execute malicious action in black-box settings. We evaluate AdapAM across eight multi-agent environments and compare it with four state-of-the-art and commonly-used baselines. Results demonstrate that AdapAM achieves the best attack performance in different perturbation rates. Besides, AdapAM-generated perturbations are the least noisy and hardest to detect, emphasizing the stealthiness.

Figures (4)

• Figure 1: The overview of our proposed AdapAM.
• Figure 2: The architecture of the Adaptive Selection Policy.
• Figure 3: The setting of the training proxy agent.
• Figure 4: The decrease in reward after being attacked at different perturbation rates, on the Normal Target MAS.