Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Surrogate Neural Networks Local Stability for Aircraft Predictive Maintenance

Mélanie Ducoffe, Guillaume Povéda, Audrey Galametz, Ryma Boumazouza, Marion-Cécile Martin, Julien Baris, Derk Daverschot, Eugene O'Higgins

TL;DR

This work tackles the safety-critical problem of verifying neural network surrogates used for aircraft predictive maintenance, specifically ensuring local stability of high-dimensional, multi-output regressors that map loads to stresses. It introduces a sequential verification pipeline that combines empirical adversarial attacks, LiRPA/CROWN incomplete bounds, and MILP-based complete verification to efficiently certify stability in the bow-tie property, with knot and wings zones defined by domain knowledge. The authors demonstrate substantial runtime gains (3–16×) over MILP-only verification on 1000 test points across multiple epoch-trained models, while also revealing model vulnerabilities and the nuanced impact of training duration on robustness. The practical impact includes a ready-to-use, open-source pipeline (AIROBAS) and guidance for integrating stability considerations into the design and certification of surrogate models for aviation maintenance.

Abstract

Surrogate Neural Networks are nowadays routinely used in industry as substitutes for computationally demanding engineering simulations (e.g., in structural analysis). They allow to generate faster predictions and thus analyses in industrial applications e.g., during a product design, testing or monitoring phases. Due to their performance and time-efficiency, these surrogate models are now being developed for use in safety-critical applications. Neural network verification and in particular the assessment of their robustness (e.g., to perturbations) is the next critical step to allow their inclusion in real-life applications and certification. We assess the applicability and scalability of empirical and formal methods in the context of aircraft predictive maintenance for surrogate neural networks designed to predict the stress sustained by an aircraft part from external loads. The case study covers a high-dimensional input and output space and the verification process thus accommodates multi-objective constraints. We explore the complementarity of verification methods in assessing the local stability property of such surrogate models to input noise. We showcase the effectiveness of sequentially combining methods in one verification 'pipeline' and demonstrate the subsequent gain in runtime required to assess the targeted property.

Surrogate Neural Networks Local Stability for Aircraft Predictive Maintenance

TL;DR

This work tackles the safety-critical problem of verifying neural network surrogates used for aircraft predictive maintenance, specifically ensuring local stability of high-dimensional, multi-output regressors that map loads to stresses. It introduces a sequential verification pipeline that combines empirical adversarial attacks, LiRPA/CROWN incomplete bounds, and MILP-based complete verification to efficiently certify stability in the bow-tie property, with knot and wings zones defined by domain knowledge. The authors demonstrate substantial runtime gains (3–16×) over MILP-only verification on 1000 test points across multiple epoch-trained models, while also revealing model vulnerabilities and the nuanced impact of training duration on robustness. The practical impact includes a ready-to-use, open-source pipeline (AIROBAS) and guidance for integrating stability considerations into the design and certification of surrogate models for aviation maintenance.

Abstract

Surrogate Neural Networks are nowadays routinely used in industry as substitutes for computationally demanding engineering simulations (e.g., in structural analysis). They allow to generate faster predictions and thus analyses in industrial applications e.g., during a product design, testing or monitoring phases. Due to their performance and time-efficiency, these surrogate models are now being developed for use in safety-critical applications. Neural network verification and in particular the assessment of their robustness (e.g., to perturbations) is the next critical step to allow their inclusion in real-life applications and certification. We assess the applicability and scalability of empirical and formal methods in the context of aircraft predictive maintenance for surrogate neural networks designed to predict the stress sustained by an aircraft part from external loads. The case study covers a high-dimensional input and output space and the verification process thus accommodates multi-objective constraints. We explore the complementarity of verification methods in assessing the local stability property of such surrogate models to input noise. We showcase the effectiveness of sequentially combining methods in one verification 'pipeline' and demonstrate the subsequent gain in runtime required to assess the targeted property.
Paper Structure (13 sections, 1 equation, 5 figures, 1 table)

This paper contains 13 sections, 1 equation, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Case study: Aircraft Loads-to-Stress Prediction
  4. Description
  5. Tested models
  6. Property to be ensured: Local stability
  7. Method combination for local stability assessment
  8. Experiments
  9. Results
  10. A significant verification time gain
  11. Insights into the models
  12. Conclusions and future work
  13. Open-source verification pipeline

Figures (5)

  • Figure 1: Verification pipeline for NN stability assessment.
  • Figure 2: Sub-pipeline included in the MILP-based approach
  • Figure 3: Percentage of successful PGD attacks for a given targeted output index (from $0$ to $80$; ranked with increasing success).
  • Figure 4: Difference of number of attacks (out of $2000$ attempts; see Fig.\ref{['fig:adv']}) between the $50$-epoch and $2000$-epoch models.
  • Figure :