A Neurosymbolic Framework for Interpretable Cognitive Attack Detection in Augmented Reality

TL;DR

This work tackles AR/MR cognitive attacks that distort semantic scene understanding, introducing CADAR, a neuro-symbolic framework that combines vision–language outputs with a symbolic perception graph and particle-filter temporal reasoning for interpretable attack detection. By converting multimodal observations into a structured spatiotemporal graph and applying probabilistic reasoning to detect semantic anomalies, CADAR achieves robust performance across diverse scenes and attack types. The authors also release CADAR-50K, the first public AR cognitive-attack dataset, and demonstrate that CADAR substantially outperforms both vision–language model baselines and traditional video classifiers, with ablations highlighting the value of the estimation module and reference-frame history. Overall, CADAR demonstrates the benefits of integrating neural perceptual foundations with symbolic, probabilistic reasoning for safe and interpretable AR systems at scale.

Abstract

Augmented Reality (AR) enriches human perception by overlaying virtual elements onto the physical world. However, this tight coupling between virtual and real content makes AR vulnerable to cognitive attacks: manipulations that distort users' semantic understanding of the environment. Existing detection methods largely focus on visual inconsistencies at the pixel or image level, offering limited semantic reasoning or interpretability. To address these limitations, we introduce CADAR, a neuro-symbolic framework for cognitive attack detection in AR that integrates neural and symbolic reasoning. CADAR fuses multimodal vision-language representations from pre-trained models into a perception graph that captures objects, relations, and temporal contextual salience. Building on this structure, a particle-filter-based statistical reasoning module infers anomalies in semantic dynamics to reveal cognitive attacks. This combination provides both the adaptability of modern vision-language models and the interpretability of probabilistic symbolic reasoning. Preliminary experiments on an AR cognitive-attack dataset demonstrate consistent advantages over existing approaches, highlighting the potential of neuro-symbolic methods for robust and interpretable AR security.

Figures (9)

• Figure 1: Illustrative AR/MR cognitive attacks. Top row: original scenes; bottom row: manipulated scenes. (a) Text Modification Attack. (b) Visual Modification attack. (c) Obstruction Attack. (d) Injection Attack.
• Figure 2: A system overview of CADAR. Sequential video frames are first transformed into spatial-temporal symbolic perception graphs. The attack-detection module then analyzes these graphs to identify, classify, and localize adversarial attacks, such as the visual modification and injection attacks shown in the examples.
• Figure 3: Symbolic perception graph generation: given a video frame and its contextual description, VLMs generate the corresponding perception graph at time step t.
• Figure 4: The particle filter-based attack detection framework consists of three main modules: matching, attack detection, and state estimation.
• Figure 5: CADAR-50K dataset overview.