Evaluation of adaptive sampling methods in scenario generation for virtual safety impact assessment of pre-crash safety systems

TL;DR

The study addresses the challenge of combinatorial explosion in virtual safety impact assessments by evaluating adaptive sampling methods for scenario generation. It compares density and severity importance sampling with an active-sampling approach, incorporating domain knowledge through adaptive sample space reduction (ASSR), stratification, and batch sampling, using a ground-truth dataset of $88{,}440$ simulations built from $44$ prototype rear-end crashes. The results show that ASSR substantially improves efficiency for both adaptive methods, with active sampling performing best when domain knowledge is limited, and stratification consistently enhancing performance; when ASSR or stratification is applied, importance sampling can match active sampling. Batch size effects reveal a trade-off between faster wall-clock times in parallel computation and higher total simulation effort, guiding practical resource allocation. Overall, incorporating domain knowledge and proper stratification markedly accelerates accurate safety impact assessments, enabling more efficient evaluation of pre-crash systems like AEB in virtual environments.

Abstract

Virtual safety assessment plays a vital role in evaluating the safety impact of pre-crash safety systems such as advanced driver assistance systems (ADAS) and automated driving systems (ADS). However, as the number of parameters in simulation-based scenario generation increases, the number of crash scenarios to simulate grows exponentially, making complete enumeration computationally infeasible. Efficient sampling methods, such as importance sampling and active sampling, have been proposed to address this challenge. However, a comprehensive evaluation of how domain knowledge, stratification, and batch sampling affect their efficiency remains limited. This study evaluates the performance of importance sampling and active sampling in scenario generation, incorporating two domain-knowledge-driven features: adaptive sample space reduction (ASSR) and stratification. Additionally, we assess the effects of a third feature, batch sampling, on computational efficiency in terms of both CPU and wall-clock time. Based on our findings, we provide practical recommendations for applying ASSR, stratification, and batch sampling to optimize sampling performance. Our results demonstrate that ASSR substantially improves sampling efficiency for both importance sampling and active sampling. When integrated into active sampling, ASSR reduces the root mean squared estimation error (RMSE) of the estimates by up to 90\%. Stratification further improves sampling performance for both methods, regardless of ASSR implementation. When ASSR and/or stratification are applied, importance sampling performs on par with active sampling, whereas when neither feature is used, active sampling is more efficient. Larger batch sizes reduce wall-clock time but increase the number of simulations required to achieve the same estimation accuracy.

Figures (8)

• Figure 1: Heatmap of the joint probability of OEOFF and deceleration, with their marginal distributions above and to the right, respectively. Glances of 0s are excluded in this plot.
• Figure 2: Empirical distributions of baseline impact speed (left) and MAIS2+ injury risk (right) in the full ground-truth dataset of simulated crashes.
• Figure 3: Flowchart illustrating the active sampling process.
• Figure 4: Root mean squared error (RMSE) for estimating (A) mean impact speed reduction, (B) crash avoidance rate, and (C) mean injury risk reduction using active sampling, compared to density and severity importance sampling. Active sampling consistently outperformed importance sampling, except for small sample sizes, where their performance was comparable. Post-stratification (case weighting) was applied to ensure equal contribution from all prototype events.
• Figure 5: Root mean squared error (RMSE) for estimating (A) mean impact speed reduction, (B) crash avoidance rate, and (C) mean injury risk reduction using active sampling with and without ASSR logic. ASSR logic provided substantial performance improvements across all settings.