Adaptive Safety Evaluation for Connected and Automated Vehicles with Sparse Control Variates

TL;DR

The paper tackles the inefficiency of evaluating CAV safety in high-dimensional driving environments by introducing sparse control variates (SCV) to reduce estimator variance without sacrificing unbiasedness. Leveraging NADE-synthesized scenarios and stratified variance reduction via multiple linear regression, the approach achieves significant efficiency gains while maintaining robustness across diverse AV models. Theoretical results establish unbiasedness and potential zero-variance conditions, while an overtaking case study demonstrates order-of-magnitude reductions in required tests and strong generalizability. This work offers a practical, scalable pathway for adaptive safety assessment of CAVs in complex, real-world-like environments.

Abstract

Safety performance evaluation is critical for developing and deploying connected and automated vehicles (CAVs). One prevailing way is to design testing scenarios using prior knowledge of CAVs, test CAVs in these scenarios, and then evaluate their safety performances. However, significant differences between CAVs and prior knowledge could severely reduce the evaluation efficiency. Towards addressing this issue, most existing studies focus on the adaptive design of testing scenarios during the CAV testing process, but so far they cannot be applied to high-dimensional scenarios. In this paper, we focus on the adaptive safety performance evaluation by leveraging the testing results, after the CAV testing process. It can significantly improve the evaluation efficiency and be applied to high-dimensional scenarios. Specifically, instead of directly evaluating the unknown quantity (e.g., crash rates) of CAV safety performances, we evaluate the differences between the unknown quantity and known quantity (i.e., control variates). By leveraging the testing results, the control variates could be well designed and optimized such that the differences are close to zero, so the evaluation variance could be dramatically reduced for different CAVs. To handle the high-dimensional scenarios, we propose the sparse control variates method, where the control variates are designed only for the sparse and critical variables of scenarios. According to the number of critical variables in each scenario, the control variates are stratified into strata and optimized within each stratum using multiple linear regression techniques. We justify the proposed method's effectiveness by rigorous theoretical analysis and empirical study of high-dimensional overtaking scenarios.

Figures (8)

• Figure 1: Illustration of the adaptive testing and evaluation framework. The focus of this study is the adaptive evaluation method for high-dimensional scenarios, where the sparse control variates method is proposed.
• Figure 2: Illustration of the sparse control variates method. The SCV are constructed by only considering critical variables (represented as red dots in testing scenarios). The testing results are stratified into strata according to the number of critical variables and then adjusted by SCV within each stratum. Finally, the performance index are obtained by summing up these evaluation results with proportion weights.
• Figure 3: Illustration of the overtaking scenarios.
• Figure 4: Crash rates of AV in NDE and NADE, where the dashed line is the crash rate estimated by NDE.
• Figure 5: RHW of AV evaluation in NDE and NADE, where the dashed line represents the RHW threshold (0.3).

Theorems & Definitions (8)

• Lemma 1
• Theorem 1
• Remark 1
• Theorem 2
• Remark 2
• Theorem 3
• Remark 3
• Remark 4