Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeepSense-V2V: A Vehicle-to-Vehicle Multi-Modal Sensing, Localization, and Communications Dataset

Joao Morais, Gouranga Charan, Nikhil Srinivas, Ahmed Alkhateeb

TL;DR

DeepSense V2V delivers the first large-scale, real-world, multi-modal V2V dataset combining mmWave communication with co-located sensing (camera, radar, LiDAR) and precise GPS. The authors describe a two-vehicle testbed, a three-stage data pipeline (collection, processing, visualization), and comprehensive statistics across diverse urban and intercity scenarios, enabling both sensing and communication research. A key demonstration shows position-based beam prediction where AoA estimated from relative vehicle positions correlates with the optimal mmWave beam, achieving robust top-$k$ accuracy across scenarios. This dataset supports exploration of sensing-aided beamforming, blockage prediction, and robust localization, with practical implications for high-rate, low-latency V2V and future 6G–beyond autonomous networks.

Abstract

High data rate and low-latency vehicle-to-vehicle (V2V) communication are essential for future intelligent transport systems to enable coordination, enhance safety, and support distributed computing and intelligence requirements. Developing effective communication strategies, however, demands realistic test scenarios and datasets. This is important at the high-frequency bands where more spectrum is available, yet harvesting this bandwidth is challenged by the need for direction transmission and the sensitivity of signal propagation to blockages. This work presents the first large-scale multi-modal dataset for studying mmWave vehicle-to-vehicle communications. It presents a two-vehicle testbed that comprises data from a 360-degree camera, four radars, four 60 GHz phased arrays, a 3D lidar, and two precise GPSs. The dataset contains vehicles driving during the day and night for 120 km in intercity and rural settings, with speeds up to 100 km per hour. More than one million objects were detected across all images, from trucks to bicycles. This work further includes detailed dataset statistics that prove the coverage of various situations and highlights how this dataset can enable novel machine-learning applications.

DeepSense-V2V: A Vehicle-to-Vehicle Multi-Modal Sensing, Localization, and Communications Dataset

TL;DR

DeepSense V2V delivers the first large-scale, real-world, multi-modal V2V dataset combining mmWave communication with co-located sensing (camera, radar, LiDAR) and precise GPS. The authors describe a two-vehicle testbed, a three-stage data pipeline (collection, processing, visualization), and comprehensive statistics across diverse urban and intercity scenarios, enabling both sensing and communication research. A key demonstration shows position-based beam prediction where AoA estimated from relative vehicle positions correlates with the optimal mmWave beam, achieving robust top- accuracy across scenarios. This dataset supports exploration of sensing-aided beamforming, blockage prediction, and robust localization, with practical implications for high-rate, low-latency V2V and future 6G–beyond autonomous networks.

Abstract

High data rate and low-latency vehicle-to-vehicle (V2V) communication are essential for future intelligent transport systems to enable coordination, enhance safety, and support distributed computing and intelligence requirements. Developing effective communication strategies, however, demands realistic test scenarios and datasets. This is important at the high-frequency bands where more spectrum is available, yet harvesting this bandwidth is challenged by the need for direction transmission and the sensitivity of signal propagation to blockages. This work presents the first large-scale multi-modal dataset for studying mmWave vehicle-to-vehicle communications. It presents a two-vehicle testbed that comprises data from a 360-degree camera, four radars, four 60 GHz phased arrays, a 3D lidar, and two precise GPSs. The dataset contains vehicles driving during the day and night for 120 km in intercity and rural settings, with speeds up to 100 km per hour. More than one million objects were detected across all images, from trucks to bicycles. This work further includes detailed dataset statistics that prove the coverage of various situations and highlights how this dataset can enable novel machine-learning applications.

Paper Structure

This paper contains 20 sections, 4 equations, 15 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. DeepSense V2V Testbed and Scenario Creation
  4. Data Collection
  5. Data Processing
  6. Data Visualization
  7. Dataset Statistics
  8. Vehicle Locations and Velocities
  9. Inter-vehicle Distance and Received Power
  10. Beam Distributions and GPS Positions
  11. Machine Vision and Image Detection
  12. Enabled Applications
  13. Wireless Communication Applications
  14. Localization
  15. Sensing Applications
  16. ...and 5 more sections

Figures (15)

  • Figure 1: DeepSense V2V testbed setup overview. For more information on the testbed visit: https://www.deepsense6g.net/data-collection/
  • Figure 2: Overview of general DeepSense structure that was used in the creation of the V2V Scenarios.
  • Figure 3: CAD design with dimensions of V2V box placement on car.
  • Figure 4: Frame of sample 4038 from video of Scenario 36. The current template shows four 90º camera views rendered around the car, the lidar pointcloud colored based on distance, four radar range-velocity plots, a GPS with the locations of the vehicles scattered on top of the satellite image of the location, and four 64-beam power vectors with the normalized received power in each beam. The video rendered for Scenario 36 data can be watched on https://www.youtube.com/watch?v=9RyZnZI7kv0&ab_channel=DeepSense6G
  • Figure 5: Satellite images with the locations of each scenario. Also included are several macro statistics of the data collection, providing contextual and objective information derived mainly from the GPS sensors.
  • ...and 10 more figures