Checklist to Define the Identification of TP, FP, and FN Object Detections in Automated Driving
Michael Hoss
TL;DR
The paper addresses the lack of a comprehensive, transparent method for identifying true positives, false positives, and false negatives in object detections for automated driving perception. It proposes a practical, criterion-based checklist that spans deterministic single-frame, temporal, and probabilistic representations to define test oracles. Two case studies demonstrate how the checklist can be applied to real-world test scenarios, improving clarity and comparability of TP/FP/FN-based metrics. While not a fully formal framework, the checklist provides a usable path toward more reliable, task-oriented safety assessment in ADS perception.
Abstract
The object perception of automated driving systems must pass quality and robustness tests before a safe deployment. Such tests typically identify true positive (TP), false-positive (FP), and false-negative (FN) detections and aggregate them to metrics. Since the literature seems to be lacking a comprehensive way to define the identification of TPs/FPs/FNs, this paper provides a checklist of relevant functional aspects and implementation details. Besides labeling policies of the test set, we cover areas of vision, occlusion handling, safety-relevant areas, matching criteria, temporal and probabilistic issues, and further aspects. Even though the checklist cannot be fully formalized, it can help practitioners minimize the ambiguity of their tests, which, in turn, makes statements on object perception more reliable and comparable.