Event-Aided Time-to-Collision Estimation for Autonomous Driving

TL;DR

This work addresses the bottleneck of frame-rate-limited TTC estimation by leveraging neuromorphic event cameras to predict Time-to-Collision with a leading vehicle. It introduces a flow-dynamics consistent, time-variant affine geometric model and a spatio-temporal registration framework based on a Linear Time Surface (LTS) representation, coupled with a robust initialization and an end-to-end forward collision warning (FCW) system. The approach achieves ultra-fast TTC estimates (up to ~200 Hz) with competitive accuracy, demonstrated on synthetic (CARLA) and real-world FCWD datasets, outperforming several baselines in both speed and precision. The work also provides a practical FCW pipeline and will release code and datasets to facilitate future research in event-based TTC estimation and autonomous driving safety.

Abstract

Predicting a potential collision with leading vehicles is an essential functionality of any autonomous/assisted driving system. One bottleneck of existing vision-based solutions is that their updating rate is limited to the frame rate of standard cameras used. In this paper, we present a novel method that estimates the time to collision using a neuromorphic event-based camera, a biologically inspired visual sensor that can sense at exactly the same rate as scene dynamics. The core of the proposed algorithm consists of a two-step approach for efficient and accurate geometric model fitting on event data in a coarse-to-fine manner. The first step is a robust linear solver based on a novel geometric measurement that overcomes the partial observability of event-based normal flow. The second step further refines the resulting model via a spatio-temporal registration process formulated as a nonlinear optimization problem. Experiments on both synthetic and real data demonstrate the effectiveness of the proposed method, outperforming other alternative methods in terms of efficiency and accuracy.

Figures (14)

• Figure 1: Illustration of the TTC estimation problem. (a) The leading vehicle is perceived by an following observer. (b) Events are triggered as the image size of the leading vehicle varies in the presence of relative speed. A motion flow field of outward expansion is visible in the close-up of the proposed image representation of events (see \ref{['subsec:Model Fitting via Spatio-Temporal Registration']} for detail), indicating a decrease in the relative distance and an increase of collision risk.
• Figure 2: Illustration of two image-like representations of event data. (a) The Time Surface with exponential decay used in zhou2021esvozhou2021emsgc; (b) The proposed Linear Time Surface. The applied color system is explained by the adjacent color bar.
• Figure 3: Comparison on characteristics of TS and LTS. (a) Events are triggered in the spatio-temporal domain as six straight lines traversing at different speeds. (b) The generated TS is an unsigned distance transform, and the gradient at the current position of contours is truncated unilaterally. (c) The generated LTS is a signed distance transform, and the gradient at the current position of contours is continuous by nature.
• Figure 4: An analysis on performing spatio-temporal registration using different measurements, including $(i)$ optical flow (OF), $(ii)$ normal flow (NF), and $(iii)$ our proposed geometric measurement, respectively. Note that only one dimension of the estimated parametric model is visualized in (b) for simplicity.
• Figure 5: Flowchart of the proposed FCW system. Key modules of the system consist of the vehicle detection module (green), the LTS rendering module (brown), the robust linear solver (red), and the spatio-temporal registration (purple).