Automated Quality Check of Sensor Data Annotations

TL;DR

This publication proposes an open-source tool designed to detect nine common errors found in multi-sensor datasets for railway vehicles, significantly reducing the manual workload and accelerating the development of these systems.

Abstract

The monitoring of the route and track environment plays an important role in automated driving. For example, it can be used as an assistance system for route monitoring in automation level Grade of Automation (GoA) 2, where the train driver is still on board. In fully automated, driverless driving at automation level GoA4, these systems finally take over environment monitoring completely independently. With the help of artificial intelligence (AI), they react automatically to risks and dangerous events on the route. To train such AI algorithms, large amounts of training data are required, which must meet high-quality standards due to their safety relevance. In this publication we present an automatic method for assuring the quality of training data, significantly reducing the manual workload and accelerating the development of these systems. We propose an open-source tool designed to detect nine common errors found in multi-sensor datasets for railway vehicles. To evaluate the performance of the framework, all detected errors were manually validated. Six issue detection methods achieved 100% precision, while three additional methods reached precision rates 96% and 97%.

Figures (4)

• Figure 1: Annotation types used in OSDaR23
• Figure 2: Issue Types covered by the automatic checks
• Figure 3: Issue types covered by the automatic checks
• Figure 4: This chart shows the observed precision of each implemented issue type detector in percent.