Security-Aware Sensor Fusion with MATE: the Multi-Agent Trust Estimator

TL;DR

The paper tackles secure situational awareness in multi-agent sensor fusion by modeling and tracking trust for each agent and tracked object. It introduces MATE, a two-part framework consisting of trust estimation (via trust pseudomeasurements mapped from perception data and Bayesian HMM updates) and trust-informed data fusion (weighting contributions by trust to maintain accurate SA under insider attacks). A novel dynamic FOV model using LiDAR ray tracing, Beta-Bernoulli conjugacy with Gibbs-like updates, and negatively-weighted trust updates enable efficient and robust trust estimation. Experiments in CARLA-based Unreal Engine smart-city environments demonstrate substantial improvements under adversarial conditions, including up to a 94% reduction in OSPA and near 90% accuracy in detecting distrusted agents, highlighting practical potential for security-aware autonomous systems.

Abstract

Lacking security awareness, sensor fusion in systems with multi-agent networks such as smart cities is vulnerable to attacks. To guard against recent threats, we design security-aware sensor fusion that is based on the estimates of distributions over trust. Trust estimation can be cast as a hidden Markov model, and we solve it by mapping sensor data to trust pseudomeasurements (PSMs) that recursively update trust posteriors in a Bayesian context. Trust then feeds sensor fusion to facilitate trust-weighted updates to situational awareness. Essential to security-awareness are a novel field of view estimator, logic to map sensor data into PSMs, and the derivation of efficient Bayesian updates. We evaluate security-aware fusion under attacks on agents using case studies and Monte Carlo simulation in the physics-based Unreal Engine simulator, CARLA. A mix of novel and classical security-relevant metrics show that our security-aware fusion enables building trustworthy situational awareness even in hostile conditions.

Figures (18)

• Figure 1: Inter-object occlusions are challenging for AVs and restrict situational awareness (SA), limiting their ability to safely traverse dynamic environments. Multi-agent fusion improves SA but can be adversarially compromised.
• Figure 2: (left) The ego suffers strong occlusion from vehicles and infrastructure. (right) Ray tracing estimates LiDAR FOV with occlusions. The predicted FOV is accurate and enables prediction of (in)visible regions.
• Figure 3: An autonomous agent negotiates a busy urban intersection with the aid of three infrastructure sensors. Colors correspond to data from each agent. Green boxes indicate tracks fused at the data aggregator. Data from all case studies are shown in Figs. \ref{['fig:results-viz-case-0']}, \ref{['fig:results-viz-case-1']}, and \ref{['fig:results-viz-case-2']}.
• Figure 4: Trust estimation and trust-informed fusion operate sequentially within a security-aware fusion architecture.
• Figure 5: Traditional sensor fusion is vulnerable to attacks on small numbers of agents even in a large multi-agent network. We introduce trust awareness into the fusion pipeline to detect, identify, mitigate, and recover from attacks to secure safety-critical infrastructure. Bayesian methods for trust awareness output distributions over trustedness for agents and tracked objects.

Theorems & Definitions (5)

• Definition 1
• Definition 2
• Definition 2.1
• Definition 2.2
• Definition 3