Consistent Valid Physically-Realizable Adversarial Attack against Crowd-flow Prediction Models

TL;DR

The paper investigates the adversarial vulnerabilities of deep crowd-flow prediction models on TaxiBJ, introducing CaV-detect to enforce Consistency and Validity of inputs and CVPR, a attack framework that is Consistent, Valid, and Physically-Realizable. It demonstrates that standard digital attacks are largely detectable by CaV-detect (FAR near 0%), while the CVPR attack can achieve high impact and evade detection, especially under adaptive threat models. The study evaluates three architectures (MLP, STResnet, TGCN) under multiple threat settings (white-box, digital and physical) and finds STResnet often strongest on clean data but most vulnerable to adaptive attacks, while TGCN shows notable robustness in adaptive scenarios. The results underscore practical limits of current defenses, highlight the importance of input validity/consistency constraints in security analyses, and point to directions for designing crowd-flow predictors with stronger intrinsic robustness against physically realizable adversarial perturbations.

Abstract

Recent works have shown that deep learning (DL) models can effectively learn city-wide crowd-flow patterns, which can be used for more effective urban planning and smart city management. However, DL models have been known to perform poorly on inconspicuous adversarial perturbations. Although many works have studied these adversarial perturbations in general, the adversarial vulnerabilities of deep crowd-flow prediction models in particular have remained largely unexplored. In this paper, we perform a rigorous analysis of the adversarial vulnerabilities of DL-based crowd-flow prediction models under multiple threat settings, making three-fold contributions. (1) We propose CaV-detect by formally identifying two novel properties - Consistency and Validity - of the crowd-flow prediction inputs that enable the detection of standard adversarial inputs with 0% false acceptance rate (FAR). (2) We leverage universal adversarial perturbations and an adaptive adversarial loss to present adaptive adversarial attacks to evade CaV-detect defense. (3) We propose CVPR, a Consistent, Valid and Physically-Realizable adversarial attack, that explicitly inducts the consistency and validity priors in the perturbation generation mechanism. We find out that although the crowd-flow models are vulnerable to adversarial perturbations, it is extremely challenging to simulate these perturbations in physical settings, notably when CaV-detect is in place. We also show that CVPR attack considerably outperforms the adaptively modified standard attacks in FAR and adversarial loss metrics. We conclude with useful insights emerging from our work and highlight promising future research directions.

Figures (17)

• Figure 1: An illustration of gridding a region and computing the inflow and the outflow matrices from the flow of crowd between adjacent regions (grid points). We typically assume the adjacency within the $2^{nd}$ neighborhood---the adjacent grid points of green highlighted area are highlighted yellow.
• Figure 2: An illustration of invalid and valid adversarial inputs generated by the standard PGD attack and our proposed CVPR attack respectively. In (a), for the grid point-$(1,0)$ highlighted green, the total (perturbed) inflow recorded is five. Three of the five inflowing devices can be C2, C3, C5 outflowing from the adjacent regions (highlighted yellow). Where are the other two devices outflowing from? In (b), CVPR attack perturbs the outflow of adjacent regions based on the inflow perturbations of the grid point-$(1,0)$ for physical plausibility.
• Figure 3: Illustrating the training setup of the crowd-flow prediction models for the TaxiBJ dataset. The trajectory data collected from the city is first converted into the inflow/outflow matrices and transformed using $\mathcal{T}$, which are then saved in the memory and concatenated with the history set to form a tuple input, $\vb{X}_h(t) = \bigcup_{i=0}^{h} \vb{x}^{t-i}$, to the crowd-flow prediction model.
• Figure 4: An illustration of the consistency property of crowd-flow state inputs. The crowd-flow state history at any time $t$, must be consistent with the crowd-flow states recorded at the previous times.
• Figure 5: Illustrating our newly proposed CaV-detect methodology integrated with the crowd-flow prediction model to detect adversarial inputs at run-time. For any input, CaV-detect checks the consistency, $\gamma_c > 0$ and the validity $\gamma_v > 0$ of the input. The input is marked adversarial if any of the checks fail. CaV-detect does not require retraining the model and can be integrated with an off-the-shelf model.