Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spatiotemporal Semantic V2X Framework for Cooperative Collision Prediction

Murat Arda Onsu, Poonam Lohan, Burak Kantarci, Aisha Syed, Matthew Andrews, Sean Kennedy

TL;DR

This work tackles proactive collision prediction in ITS under V2X bandwidth and latency constraints by shifting from raw video transmission to semantic, predictive spatiotemporal embeddings. It deploys V-JEPA at RSUs to generate future-frame embeddings, which are transmitted to vehicles and decoded by a lightweight attentive probe and classifier to predict imminent collisions. Using a Quanser QLabs digital twin, the approach demonstrates substantial bandwidth savings (up to five orders of magnitude reduction) while achieving high predictive accuracy (around 92% and up to 8% F1-score gains). The results validate semantic V2X as a practical pathway to real-time, cooperative collision anticipation in urban environments.

Abstract

Intelligent Transportation Systems (ITS) demand real-time collision prediction to ensure road safety and reduce accident severity. Conventional approaches rely on transmitting raw video or high-dimensional sensory data from roadside units (RSUs) to vehicles, which is impractical under vehicular communication bandwidth and latency constraints. In this work, we propose a semantic V2X framework in which RSU-mounted cameras generate spatiotemporal semantic embeddings of future frames using the Video Joint Embedding Predictive Architecture (V-JEPA). To evaluate the system, we construct a digital twin of an urban traffic environment enabling the generation of d verse traffic scenarios with both safe and collision events. These embeddings of the future frame, extracted from V-JEPA, capture task-relevant traffic dynamics and are transmitted via V2X links to vehicles, where a lightweight attentive probe and classifier decode them to predict imminent collisions. By transmitting only semantic embeddings instead of raw frames, the proposed system significantly reduces communication overhead while maintaining predictive accuracy. Experimental results demonstrate that the framework with an appropriate processing method achieves a 10% F1-score improvement for collision prediction while reducing transmission requirements by four orders of magnitude compared to raw video. This validates the potential of semantic V2X communication to enable cooperative, real-time collision prediction in ITS.

Spatiotemporal Semantic V2X Framework for Cooperative Collision Prediction

TL;DR

This work tackles proactive collision prediction in ITS under V2X bandwidth and latency constraints by shifting from raw video transmission to semantic, predictive spatiotemporal embeddings. It deploys V-JEPA at RSUs to generate future-frame embeddings, which are transmitted to vehicles and decoded by a lightweight attentive probe and classifier to predict imminent collisions. Using a Quanser QLabs digital twin, the approach demonstrates substantial bandwidth savings (up to five orders of magnitude reduction) while achieving high predictive accuracy (around 92% and up to 8% F1-score gains). The results validate semantic V2X as a practical pathway to real-time, cooperative collision anticipation in urban environments.

Abstract

Intelligent Transportation Systems (ITS) demand real-time collision prediction to ensure road safety and reduce accident severity. Conventional approaches rely on transmitting raw video or high-dimensional sensory data from roadside units (RSUs) to vehicles, which is impractical under vehicular communication bandwidth and latency constraints. In this work, we propose a semantic V2X framework in which RSU-mounted cameras generate spatiotemporal semantic embeddings of future frames using the Video Joint Embedding Predictive Architecture (V-JEPA). To evaluate the system, we construct a digital twin of an urban traffic environment enabling the generation of d verse traffic scenarios with both safe and collision events. These embeddings of the future frame, extracted from V-JEPA, capture task-relevant traffic dynamics and are transmitted via V2X links to vehicles, where a lightweight attentive probe and classifier decode them to predict imminent collisions. By transmitting only semantic embeddings instead of raw frames, the proposed system significantly reduces communication overhead while maintaining predictive accuracy. Experimental results demonstrate that the framework with an appropriate processing method achieves a 10% F1-score improvement for collision prediction while reducing transmission requirements by four orders of magnitude compared to raw video. This validates the potential of semantic V2X communication to enable cooperative, real-time collision prediction in ITS.
Paper Structure (19 sections, 7 equations, 5 figures, 2 tables)

This paper contains 19 sections, 7 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Dataset Generation
  5. Video Generation using QLabs
  6. Video Post-Processing
  7. Semantic Encoder: V-JEPA at RSU
  8. V-JEPA Pretraining with Masked Modeling
  9. Fine-Tuning with Attentive Probe
  10. Semantic Decoder at Vehicle
  11. Transmission Efficiency in V2X Semantic Communication
  12. Complexity Analysis of Frozen Encoder with Attentive Probe
  13. Frozen Encoder Cost
  14. Attentive Probe and Classifier Cost
  15. End-to-End Computational Cost
  16. ...and 4 more sections

Figures (5)

  • Figure 1: Data Collection Using Digital-Twin (Quanser Interactive Lab)
  • Figure 2: Different Post-processing Methods for Video Dataset
  • Figure 3: V-JEPA Pretraining and Fine-Tuning for Collision Prediction
  • Figure 4: Average compression ratio for different video lengths and Tx latency for different modulation schemes
  • Figure 5: Results for Different Post-Processing Techniques