OpenCat: Improving Interoperability of ADS Testing

TL;DR

OpenCat provides a dedicated converter from OpenDRIVE to Catmull-Rom splines to address interoperability gaps in ADAS benchmarks like SensoDat. The authors demonstrate 100% geometric fidelity ($R^2=1$) when converting 32,580 roads and validate practical usefulness by evaluating a lane-keeping ADAS in the Udacity simulator, revealing that benchmarks tied to a specific ADAS limit broader reuse. The results show a dramatic improvement in test pass rates (about 98% vs 61%), underscoring the value of architecture-agnostic benchmarks for meaningful regression testing across simulators and models. Together, the work advocates decoupling benchmarks from particular ADAS implementations to enhance generalizability and provides concrete steps to extend road-interoperability in ADAS evaluation.

Abstract

Testing Advanced Driving Assistance Systems (ADAS), such as lane-keeping functions, requires creating road topologies or using predefined benchmarks. However, the test cases in existing ADAS benchmarks are often designed in specific formats (e.g., OpenDRIVE) and tailored to specific ADAS models. This limits their reusability and interoperability with other simulators and models, making it challenging to assess ADAS functionalities independently of the platform-specific details used to create the test cases. This paper evaluates the interoperability of SensoDat, a benchmark developed for ADAS regression testing. We introduce OpenCat, a converter that transforms OpenDRIVE test cases into the Catmull-Rom spline format, which is widely supported by many current test generators. By applying OpenCat to the SensoDat dataset, we achieved high accuracy in converting test cases into reusable road scenarios. To validate the converted scenarios, we used them to evaluate a lane-keeping ADAS model using the Udacity simulator. Both the simulator and the ADAS model operate independently of the technologies underlying SensoDat, ensuring an unbiased evaluation of the original test cases. Our findings reveal that benchmarks built with specific ADAS models hinder their effective usage for regression testing. We conclude by offering insights and recommendations to enhance the reusability and transferability of ADAS benchmarks for more extensive applications.

Figures (4)

• Figure 1: Elements of OpenDRIVE asam_opendrive.
• Figure 2: Catmull-Rom spline interpolation with four points.
• Figure 3: Conversion process from OpenDRIVE format to Catmull-Rom spline.
• Figure 4: Comparative view of OpenDRIVE road, generated spline, and test case.