Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

What, Indeed, is an Achievable Provable Guarantee for Learning-Enabled Safety Critical Systems

Saddek Bensalem, Chih-Hong Cheng, Wei Huang, Xiaowei Huang, Changshun Wu, Xingyu Zhao

TL;DR

This paper discusses the engineering and research challenges associated with the design and verification of safety guarantees in machine learning systems, and promotes a two-step verification method for the ultimate achievement of provable statistical guarantees.

Abstract

Machine learning has made remarkable advancements, but confidently utilising learning-enabled components in safety-critical domains still poses challenges. Among the challenges, it is known that a rigorous, yet practical, way of achieving safety guarantees is one of the most prominent. In this paper, we first discuss the engineering and research challenges associated with the design and verification of such systems. Then, based on the observation that existing works cannot actually achieve provable guarantees, we promote a two-step verification method for the ultimate achievement of provable statistical guarantees.

What, Indeed, is an Achievable Provable Guarantee for Learning-Enabled Safety Critical Systems

TL;DR

This paper discusses the engineering and research challenges associated with the design and verification of safety guarantees in machine learning systems, and promotes a two-step verification method for the ultimate achievement of provable statistical guarantees.

Abstract

Machine learning has made remarkable advancements, but confidently utilising learning-enabled components in safety-critical domains still poses challenges. Among the challenges, it is known that a rigorous, yet practical, way of achieving safety guarantees is one of the most prominent. In this paper, we first discuss the engineering and research challenges associated with the design and verification of such systems. Then, based on the observation that existing works cannot actually achieve provable guarantees, we promote a two-step verification method for the ultimate achievement of provable statistical guarantees.
Paper Structure (18 sections, 7 equations, 3 figures)

This paper contains 18 sections, 7 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Challenges in Engineering Safety-critical Systems integrating Learning-enabled Components
  3. Research Challenges
  4. Uncertainty
  5. Size and Complexity of the AI Models
  6. Lack of Novel Analysis Methods that are both Rigorous and Efficient
  7. Methodology
  8. State-of-the-Art 1: A Design and Co-Simulation Framework
  9. Design of trustable AI models
  10. Design flow for safety-critical systems with AI components
  11. Simulation-based Modeling and Testing at Design Time
  12. Deployment, Operation, and Analysis of AI critical systems at Runtime
  13. Properties and Specifications at Different Levels
  14. Guarantees Achieved at Component Levels
  15. State-Of-The-Art 2: Offline V&V Methods and Guarantee
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Research challenges organised into five conceptual levels with top-down and bottom-up routes.
  • Figure 2: A Verification and Validation Framework for Machine Learning Enhancement
  • Figure 3: Runtime Monitors with (for non-ML systems) and without (for ML systems) specifications

Theorems & Definitions (3)

  • remark thmcounterremark
  • definition thmcounterdefinition
  • remark thmcounterremark