Are you a robot? Detecting Autonomous Vehicles from Behavior Analysis

TL;DR

The paper tackles the problem of identifying autonomous vehicles within mixed traffic by analyzing target vehicle behavior through multivariate time-series of TV state and 2D bounding-box detections from camera data. It introduces NexusStreets, a publicly released dataset generated in CARLA with Baidu Apollo autonomous control and human-driven driving, enabling supervised classification of TV autonomy. Through RF and LSTM models, it shows that state-history and detections—especially when combined—achieve high discriminative performance (auROC near or above 0.95), with robustness analyses under data degradation. The work also explores future-state autoregression and discusses risk-aware training directions and real-world data integration, highlighting practical pathways to tune autonomous systems via shared telemetry.

Abstract

The tremendous hype around autonomous driving is eagerly calling for emerging and novel technologies to support advanced mobility use cases. As car manufactures keep developing SAE level 3+ systems to improve the safety and comfort of passengers, traffic authorities need to establish new procedures to manage the transition from human-driven to fully-autonomous vehicles while providing a feedback-loop mechanism to fine-tune envisioned autonomous systems. Thus, a way to automatically profile autonomous vehicles and differentiate those from human-driven ones is a must. In this paper, we present a fully-fledged framework that monitors active vehicles using camera images and state information in order to determine whether vehicles are autonomous, without requiring any active notification from the vehicles themselves. Essentially, it builds on the cooperation among vehicles, which share their data acquired on the road feeding a machine learning model to identify autonomous cars. We extensively tested our solution and created the NexusStreet dataset, by means of the CARLA simulator, employing an autonomous driving control agent and a steering wheel maneuvered by licensed drivers. Experiments show it is possible to discriminate the two behaviors by analyzing video clips with an accuracy of 80%, which improves up to 93% when the target state information is available. Lastly, we deliberately degraded the state to observe how the framework performs under non-ideal data collection conditions.

Figures (11)

• Figure 1: Envisioned vehicular communication scenario. Vehicles report state information and collectively identify autonomous cars.
• Figure 2: Illustration of the Autonomous Vehicle Detection Methodology
• Figure 3: Schematic illustration of the carla-apollo bridge architecture
• Figure 4: $T$ and $E$ are the two reference systems for TV and EV, respectively. Their origins lay in their center of mass. For simplicity, we only depict the $x$- and $y$-axis, however the $z$-axis is also considered in the simulations. The TV state is constituted by its position vector $\bm{d}$, speed $\dot{\bm{d}}$ and acceleration $\ddot{\bm{d}}$ w.r.t. the $E$ frame, as well as the distance from the lane center $l$ and its yaw angle $\omega$ calculated in $T$. Moreover, we use the reference system $F$ to represent the 2D bounding box coordinates. $F(0,0)$ corresponds to the upper left pixel of each 960x540 JPEG image acquired by the EV camera.
• Figure 5: Distribution of each TV state feature, highlighting $25$th quartile, median, $75$th quartile, and values falling outside such two boundaries.