A Safety-Adapted Loss for Pedestrian Detection in Automated Driving

TL;DR

The paper tackles the problem of safety-critical misses in pedestrian detection for automated driving by introducing a safety-adapted focal loss that uses per-pedestrian criticality derived from motion-domain reachability ($TTC_{RSB}$) and distance to the vehicle. It combines collision risk with distance into a single criticality weight $\kappa_i$ and modulates the focal loss exponent as $\gamma-\kappa_i$, ensuring重点 attention to hazardous pedestrians while preserving overall performance. Evaluations on nuScenes with RetinaNet and FCOS show meaningful recall improvements in safety-critical zones (approximately $2.5$–$3.2$ percentage points) with negligible degradation in general metrics like AP$^{50}$, demonstrating practical gains for AD safety. The work also includes ablations revealing the relative importance of TTC-based versus distance-based criticality and discusses avenues to address false positives and extend the framework to regression tasks.

Abstract

In safety-critical domains like automated driving (AD), errors by the object detector may endanger pedestrians and other vulnerable road users (VRU). As common evaluation metrics are not an adequate safety indicator, recent works employ approaches to identify safety-critical VRU and back-annotate the risk to the object detector. However, those approaches do not consider the safety factor in the deep neural network (DNN) training process. Thus, state-of-the-art DNN penalizes all misdetections equally irrespective of their criticality. Subsequently, to mitigate the occurrence of critical failure cases, i.e., false negatives, a safety-aware training strategy might be required to enhance the detection performance for critical pedestrians. In this paper, we propose a novel safety-aware loss variation that leverages the estimated per-pedestrian criticality scores during training. We exploit the reachability set-based time-to-collision (TTC-RSB) metric from the motion domain along with distance information to account for the worst-case threat quantifying the criticality. Our evaluation results using RetinaNet and FCOS on the nuScenes dataset demonstrate that training the models with our safety-aware loss function mitigates the misdetection of critical pedestrians without sacrificing performance for the general case, i.e., pedestrians outside the safety-critical zone.

Figures (7)

• Figure 1: We propose the safety-adapted focal loss to enhance the detection performance of pedestrians under risk. The loss diligently exploits distance information and a threat metric from the motion domain (BEV) to consider the criticality of individual pedestrians during the training of an one-stage 2D object detector (vision domain). Concretely, we leverage reachable set-based time-to-collision ($TTC_{RSB}$), defined as the intersection between the AV's (yellow) and pedestrians' (red) reachable sets, to quantify the per-pedestrian criticality for each pedestrian in the scene. We back-annotate the criticality scores of the detections to the safety-adapted loss function and to dynamically adjust the loss contribution of safety-critical pedestrians.
• Figure 2: Exemplary illustration of a safety-critical interaction at time $\tau$ between an AV and pedestrians at risk (blue markers) and the respective reachable sets $R_{AV}(\tau)$ and $R_{ped,i}(\tau)$ for a given AV trajectory (black dots). The intersection of the reachable sets emphasizes the collision risk with $TTC_{RSB}\,<\,\infty$, i.e., $R_{AV}(\tau) \cap R_{ped}(\tau) \neq \emptyset\ $. To calculate $R_{AV} (\tau)$w.r.t. the planned driving corridor Schneider2021, we require a sequence of the current (red) and successive centerlines (orange) as the input to the reachability framework from LGH+22 to define our road.
• Figure 3: Downward parabola to model the distance criticality $\kappa_{d,i}$ over the distance $d_i$ to the AV for a pedestrian $i$.
• Figure 4: Illustration of the focal loss FL (with $\gamma=2, \alpha=0.25$) and the safety-adapted focal loss $FL_{\kappa,i}$w.r.t. the pedestrians' criticality $\kappa_i$. For $\kappa_{i}=0$ (non-critical pedestrians), we preserve the properties of the FL, i.e., $FL_{\kappa,i}\xrightarrow~FL$. For critical pedestrians with $\kappa\xrightarrow~1$, we naturally magnify the loss contribution, where $FL_{\kappa\xrightarrow~1}\geq~FL$.
• Figure 5: Invalid 2D bounding box due to nuScenes' annotation policy.