Accident Anticipation via Temporal Occurrence Prediction

TL;DR

This paper addresses accident anticipation by moving from frame-level risk scores to predicting a sequence of accident scores for multiple future horizons, using precise accident timestamps as supervision. It introduces a snippet-level encoder plus a Transformer-based temporal decoder to output scores for horizons from $0.1$ s to $2.0$ s ahead, trained with a weighted Binary Cross-Entropy loss on ground-truth accident times. A refined evaluation protocol with FAR-aware metrics (AUC$^\lambda$, mAUC$^\lambda$, mTTA$^\lambda$) and horizon-specific recalls, plus an improved Time-To-Accident calculation, ensures practical applicability. Experiments on MM-AU CAP CAP and DADA demonstrate state-of-the-art performance, particularly near crashes, while acknowledging challenges at longer horizons and emphasizing real-world safety considerations.

Abstract

Accident anticipation aims to predict potential collisions in an online manner, enabling timely alerts to enhance road safety. Existing methods typically predict frame-level risk scores as indicators of hazard. However, these approaches rely on ambiguous binary supervision (labeling all frames in accident videos as positive) despite the fact that risk varies continuously over time, leading to unreliable learning and false alarms. To address this, we propose a novel paradigm that shifts the prediction target from current-frame risk scoring to directly estimating accident scores at multiple future time steps (e.g., 0.1s-2.0s ahead), leveraging precisely annotated accident timestamps as supervision. Our method employs a snippet-level encoder to jointly model spatial and temporal dynamics, and a Transformer-based temporal decoder that predicts accident scores for all future horizons simultaneously using dedicated temporal queries. Furthermore, we introduce a refined evaluation protocol that reports Time-to-Accident (TTA) and recall (evaluated at multiple pre-accident intervals (0.5s, 1.0s, and 1.5s)) only when the false alarm rate (FAR) remains within an acceptable range, ensuring practical relevance. Experiments show that our method achieves superior performance in both recall and TTA under realistic FAR constraints.

Figures (5)

• Figure 1: Comparison of accident anticipation paradigms. (a) Previous works predict a single risk score for the current frame, which is ambiguous and hard to supervise accurately. (b) Our method predicts a sequence of accident scores at multiple future time steps (e.g., 0.1s, 0.2s, …, 2.0s ahead), where each score indicates the likelihood of an accident occurring exactly at that future time.
• Figure 2: Overview of our encoder-decoder method. We feed the observed frames into the snippet encoder and use the temporal decoder to predict accident scores for unobserved future frames at multiple time steps, where the accident score label is 1 for the accident frame and 0 otherwise.
• Figure 3: Comparison of the TTA calculation approaches between previous works and our method. We only compute TTA after the anomaly appears, because the moment the anomaly appears is the earliest time annotated by humans as being predictive of the accident. If a model issues an alert earlier than this moment, the perceived risk is typically unrelated to the actual accident that eventually occurs. We consider such alerts as false alarms rather than early warnings. However, previous works did not account for this when calculating TTA, leading to inflated TTA values—sometimes even exceeding 3 seconds.
• Figure 4: ROC curves of our method on CAP CAP at different accident anticipation horizons: 0.0s, 0.5s, 1.0s, and 1.5s before the accident.
• Figure 5: Qualitative results on CAP CAP. Each case shows the trend of the maximum accident score predicted over future time steps; an alarm is triggered if this maximum exceeds the threshold.