Malicious Agent Detection for Robust Multi-Agent Collaborative Perception

TL;DR

This work addresses adversarial threats in multi-agent collaborative (MAC) perception by introducing Malicious Agent Detection (MADE), a reactive defense that detects and removes malicious agents in the ego agent’s collaboration network. MADE uses a two-test framework with conformal p-values and the Benjamini-Hochberg procedure, leveraging a match loss for output consistency and a collaborative reconstruction loss for intermediate-feature consistency to identify inconsistencies relative to the ego agent. The approach is evaluated on V2X-SIM and DAIR-V2X, showing that MADE substantially mitigates performance degradation from attacks and approaches Oracle-level robustness, outperforming prior defenses like adversarial training and ROBOSAC. The results demonstrate both effectiveness and resilience to adaptive and unsupervised threat scenarios, with practical implications for safer MAC perception in safety-critical applications.

Abstract

Recently, multi-agent collaborative (MAC) perception has been proposed and outperformed the traditional single-agent perception in many applications, such as autonomous driving. However, MAC perception is more vulnerable to adversarial attacks than single-agent perception due to the information exchange. The attacker can easily degrade the performance of a victim agent by sending harmful information from a malicious agent nearby. In this paper, we extend adversarial attacks to an important perception task -- MAC object detection, where generic defenses such as adversarial training are no longer effective against these attacks. More importantly, we propose Malicious Agent Detection (MADE), a reactive defense specific to MAC perception that can be deployed by each agent to accurately detect and then remove any potential malicious agent in its local collaboration network. In particular, MADE inspects each agent in the network independently using a semi-supervised anomaly detector based on a double-hypothesis test with the Benjamini-Hochberg procedure to control the false positive rate of the inference. For the two hypothesis tests, we propose a match loss statistic and a collaborative reconstruction loss statistic, respectively, both based on the consistency between the agent to be inspected and the ego agent where our detector is deployed. We conduct comprehensive evaluations on a benchmark 3D dataset V2X-sim and a real-road dataset DAIR-V2X and show that with the protection of MADE, the drops in the average precision compared with the best-case "oracle" defender against our attack are merely 1.28% and 0.34%, respectively, much lower than 8.92% and 10.00% for adversarial training, respectively.

Figures (2)

• Figure 1: Illustration of a standard MAC perception pipeline protected by our proposed MADE. Message (i.e. intermediate feature map) from each agent will be inspected by MADE before fusing -- malicious agents will be removed once detected by MADE. Specifically, for any agent $a_i$ (with a feature map $F_i$) to be inspected, we obtain $Z_{\rm ego}$ directly from the ego agent's feature map $F_{\rm ego}$ (without fusion) and obtain a fused feature map $Z_{{\rm ego}+i}$ by fusing $F_i$ with $F_{\rm ego}$. We then obtain the bounding box proposals $Y_{\rm ego}$ and $Y_{{\rm ego}+i}$ from $Z_{\rm ego}$ and $Z_{{\rm ego}+i}$, respectively. A collaborative reconstruction loss is computed on $Z_{\rm ego}$ and $Z_{{\rm ego}+i}$, while a match loss statistic is computed on $Y_{\rm ego}$ and $Y_{{\rm ego}+i}$. Finally, conformal p-values are computed for the two tests respectively, which are then used for a multi-test with the BH procedure, leading to the inference result for agent $a_i$.
• Figure 2: Comparative visualization of intermediate feature maps and detection outputs with and without adversarial attack