Optimizing Falsification for Learning-Based Control Systems: A Multi-Fidelity Bayesian Approach

TL;DR

This paper proposes a multi-fidelity Bayesian optimization falsification framework that harnesses simulators with varying levels of accuracy and demonstrates that it is more computationally efficient than full-fidelity Bayesian optimization and other baseline methods in detecting counterexamples.

Abstract

Testing controllers in safety-critical systems is vital for ensuring their safety and preventing failures. In this paper, we address the falsification problem within learning-based closed-loop control systems through simulation. This problem involves the identification of counterexamples that violate system safety requirements and can be formulated as an optimization task based on these requirements. Using full-fidelity simulator data in this optimization problem can be computationally expensive. To improve efficiency, we propose a multi-fidelity Bayesian optimization falsification framework that harnesses simulators with varying levels of accuracy. Our proposed framework can transition between different simulators and establish meaningful relationships between them. Through multi-fidelity Bayesian optimization, we determine both the optimal system input likely to be a counterexample and the appropriate fidelity level for assessment. We evaluated our approach across various Gym environments, each featuring different levels of fidelity. Our experiments demonstrate that multi-fidelity Bayesian optimization is more computationally efficient than full-fidelity Bayesian optimization and other baseline methods in detecting counterexamples. A Python implementation of the algorithm is available at https://github.com/SAILRIT/MFBO_Falsification.

Figures (9)

• Figure 1: Overview of multi-fidelity BO falsification. The algorithm begins by performing random experiments at different fidelity levels, with fewer experiments conducted at higher fidelity levels. The robustness values across trajectories are recorded, and a GP model is initialized. The algorithm then optimizes entropy search over this model to select a candidate environment parameter $\mathbf{e}$, along with the fidelity index $i\in \left \{1,\ldots ,q \right \}$ at which the next experiment should be performed. The GP model is then updated with the new data.
• Figure 2: Fidelity levels in the highway case study. (a) Low-fidelity highway simulator at $t=t_{0}$ and $t=12t_{0}$. (b) High-fidelity highway simulator at $t=t_{0}$ and $t>16t_{0}$. The simulation frequencies in the low-fidelity simulator and the high-fidelity simulator are $11$ and $15$; respectively. For an identical scenario, the high-fidelity simulator is slower. Additionally, the high-fidelity simulator features more cars on the road.
• Figure 3: Average reliability percentage of discovered counterexamples over $750$ tests using standard BO on low- and middle-fidelity simulators, two-fidelity BO and three-fidelity BO methods for cart-pole, lunar lander, highway, merge, and roundabout case studies.
• Figure 4: Comparison between average cost of finding a counterexample through $200$ BO iterations on cart-pole case study.
• Figure 5: Comparison between average cost of finding a counterexample through $200$ BO iterations on lunar lander case study.