EvTTC: An Event Camera Dataset for Time-to-Collision Estimation

TL;DR

The paper tackles TTC estimation for FCW/AEB in autonomous driving by addressing latency limitations of frame cameras with a novel EvTTC dataset that leverages event cameras alongside RGB, LiDAR, and GNSS/INS ground truth. It introduces a synchronized, dual RGB–Event sensor setup at two focal lengths, a high-fidelity ground-truth pipeline, and a small-scale TTC testbed to enable affordable, rapid experimentation. Ground-truth TTC is computed as $TTC = \frac{Z}{V_{rel}}$ in the camera frame, with depth and pose provided by LiDAR and GNSS/INS and refined via LiDAR odometry and occlusion handling. The work provides a comprehensive benchmark and open-source resources to accelerate development of vision-based TTC methods under high-speed and complex scenarios.

Abstract

Time-to-Collision (TTC) estimation lies in the core of the forward collision warning (FCW) functionality, which is key to all Automatic Emergency Braking (AEB) systems. Although the success of solutions using frame-based cameras (e.g., Mobileye's solutions) has been witnessed in normal situations, some extreme cases, such as the sudden variation in the relative speed of leading vehicles and the sudden appearance of pedestrians, still pose significant risks that cannot be handled. This is due to the inherent imaging principles of frame-based cameras, where the time interval between adjacent exposures introduces considerable system latency to AEB. Event cameras, as a novel bio-inspired sensor, offer ultra-high temporal resolution and can asynchronously report brightness changes at the microsecond level. To explore the potential of event cameras in the above-mentioned challenging cases, we propose EvTTC, which is, to the best of our knowledge, the first multi-sensor dataset focusing on TTC tasks under high-relative-speed scenarios. EvTTC consists of data collected using standard cameras and event cameras, covering various potential collision scenarios in daily driving and involving multiple collision objects. Additionally, LiDAR and GNSS/INS measurements are provided for the calculation of ground-truth TTC. Considering the high cost of testing TTC algorithms on full-scale mobile platforms, we also provide a small-scale TTC testbed for experimental validation and data augmentation. All the data and the design of the testbed are open sourced, and they can serve as a benchmark that will facilitate the development of vision-based TTC techniques.

Figures (8)

• Figure -1: Illustration of our synchronization scheme.
• Figure 0: Illustration of the CAD model of the sensor suite. The axes of all sensors are labeled and color-coded as follows: red for X, green for Y, and blue for Z.
• Figure 1: The top-view schematic of the dataset scenarios. The arrow represents the direction of movement. The lateral shadowing of the DCV in the CCRs and CCRm scenarios indicates that data are collected across different lane positions. The objects in the scene include the Global Vehicle Target (GVT), Adult Pedestrian Target (APT), and Data Collection Vehicle (DCV).
• Figure 2: Illustration of real-world road scenes in our dataset. The top of the first and second rows respectively shows the RGB images for each scenario, while the bottom of the first and second rows respectively presents the accumulated event data for each scenario.
• Figure 3: Parameter definitions in TTC Scenarios.$\Delta \text{T}_1\text{(s)}$: duration at constant speed, $\Delta \text{T}_2\text{(s)}$: braking duration, $\text{D}_1\text{(m)}$: distance to the collision target when the DCV reaches maximum speed, $\text{D}_2\text{(m)}$: distance to the collision target at braking onset.