High-Dimensional Fault Tolerance Testing of Highly Automated Vehicles Based on Low-Rank Models

TL;DR

This work reframes fault-injection testing for Highly Automated Vehicles as a high-dimensional, sparse matrix-completion problem. It introduces a low-rank matrix factorization framework, SRMF, augmented with three types of smoothness regularization to exploit correlations across fault values, scenarios, and fault-injection times, and to generalize to new scenarios and rare faults. The method achieves substantial acceleration (up to ~1171×) while delivering high accuracy and reliability in identifying critical faults (precision ~99.3% and F1 ~91.1%), outperforming several surrogate baselines. The study demonstrates strong potential for efficient FI testing in HAV development, with clear avenues for scaling to larger scenario/fault spaces and integration with other safety-assessment approaches.

Abstract

Ensuring fault tolerance of Highly Automated Vehicles (HAVs) is crucial for their safety due to the presence of potentially severe faults. Hence, Fault Injection (FI) testing is conducted by practitioners to evaluate the safety level of HAVs. To fully cover test cases, various driving scenarios and fault settings should be considered. However, due to numerous combinations of test scenarios and fault settings, the testing space can be complex and high-dimensional. In addition, evaluating performance in all newly added scenarios is resource-consuming. The rarity of critical faults that can cause security problems further strengthens the challenge. To address these challenges, we propose to accelerate FI testing under the low-rank Smoothness Regularized Matrix Factorization (SRMF) framework. We first organize the sparse evaluated data into a structured matrix based on its safety values. Then the untested values are estimated by the correlation captured by the matrix structure. To address high dimensionality, a low-rank constraint is imposed on the testing space. To exploit the relationships between existing scenarios and new scenarios and capture the local regularity of critical faults, three types of smoothness regularization are further designed as a complement. We conduct experiments on car following and cut in scenarios. The results indicate that SRMF has the lowest prediction error in various scenarios and is capable of predicting rare critical faults compared to other machine learning models. In addition, SRMF can achieve 1171 acceleration rate, 99.3% precision and 91.1% F1 score in identifying critical faults. To the best of our knowledge, this is the first work to introduce low-rank models to FI testing of HAVs.

Figures (7)

• Figure 1: Illustration of fault injection testing problem. The high dimensional testing space consists of scenario parameters and fault parameters. Only a few testing results can be obtained, which compose the sparse data. However, predicting new scenarios and rare critical faults may be difficult.
• Figure 2: Data organization and smoothness regularization. (1) There are $K$ concrete scenarios to be tested in total, with $P_1, P_2, ...,P_k$ functional scenarios marked in different colors. In each concrete scenario, there are $I$ types of fault value and $J$ types of fault injection time. Only a subset of concrete scenarios and only a subset of faults are tested, with the darker colors indicating the tested ones. (2) Then, the data is transformed into a matrix $\boldsymbol{X}\in\mathbb{R}^{(JK)\times I}$. Three kinds of smoothness regularization are applied. The red arrows represent the smoothness regularization in rows, connecting different fault value. The dark blue arrows and the arrows drawn at the top are used in columns to capture the relationship between scenarios and fault injection time.
• Figure 3: Car following and cut in scenario.
• Figure 4: Simulation and prediction results of cut in and car following scenario. The number colored in red are the scenario id and each grid divided by red dotted lines represents a scenario with $50*50$ kinds of faults. The X-axis represents fault injection time and Y-axis represents fault value. Safety indicator of each fault is shown by the color. The darker the color, the more serious the fault. The precision of each model is also marked.
• Figure 5: Comparison between , MLP and XGB in the cut in and car following scenario. In the first figure, the critical fault rate gradually decreases and the scenarios with rare critical faults are marked in red. The second to fifth figures are the MAE, WMAPE, Precision, F1 score of , MLP and XGB in different scenarios. The MAE, WMAPE of (colored orange) is smaller than MLP (colored blue) and XGB (colored green), and the Precision and F1 score of is higher than MLP and XGB overall.