Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Abort Policy for Mission-Critical Systems under Imperfect Condition Monitoring

Qiuzhuang Sun, Jiawen Hu, Zhi-Sheng Ye

TL;DR

This work develops a general framework for optimal mission abort decisions under imperfect condition monitoring when deterioration follows a three-state semi-Markov process. By approximating non-exponential sojourn times with Erlang mixtures, the authors build surrogate CTMCs and a surrogate POMDP whose solution converges to the true optimum as the Erlang rate grows. They prove structural properties: the value function is piecewise linear and concave, with a convex abort region that, in spherical coordinates, reduces to radius-based control limits that depend on the angle. A modified PBVI algorithm exploiting these structures enables real-time-ish computation, and extensions to multi-task missions and UAV case studies demonstrate significant cost savings over rule-based and simple CTMC benchmarks. The framework provides asymptotically optimal abort policies for non-Markovian systems and offers a practical approach to robust, data-driven mission abort decisions. The results have direct implications for safety-critical autonomous systems and CBM-like settings where non-Markovian dynamics and imperfect sensing are prevalent.

Abstract

While most on-demand mission-critical systems are engineered to be reliable to support critical tasks, occasional failures may still occur during missions. To increase system survivability, a common practice is to abort the mission before an imminent failure. We consider optimal mission abort for a system whose deterioration follows a general three-state (normal, defective, failed) semi-Markov chain. The failure is assumed self-revealed, while the healthy and defective states have to be {inferred} from imperfect condition monitoring data. Due to the non-Markovian process dynamics, optimal mission abort for this partially observable system is an intractable stopping problem. For a tractable solution, we introduce a novel tool of Erlang mixtures to approximate non-exponential sojourn times in the semi-Markov chain. This allows us to approximate the original process by a surrogate continuous-time Markov chain whose optimal control policy can be solved through a partially observable Markov decision process (POMDP). We show that the POMDP optimal policies converge almost surely to the optimal abort decision rules when the Erlang rate parameter diverges. This implies that the expected cost by adopting the POMDP solution converges to the optimal expected cost. Next, we provide comprehensive structural results on the optimal policy of the surrogate POMDP. Based on the results, we develop a modified point-based value iteration algorithm to numerically solve the surrogate POMDP. We further consider mission abort in a multi-task setting where a system executes several tasks consecutively before a thorough inspection. Through a case study on an unmanned aerial vehicle, we demonstrate the capability of real-time implementation of our model, even when the condition-monitoring signals are generated with high frequency.

Optimal Abort Policy for Mission-Critical Systems under Imperfect Condition Monitoring

TL;DR

This work develops a general framework for optimal mission abort decisions under imperfect condition monitoring when deterioration follows a three-state semi-Markov process. By approximating non-exponential sojourn times with Erlang mixtures, the authors build surrogate CTMCs and a surrogate POMDP whose solution converges to the true optimum as the Erlang rate grows. They prove structural properties: the value function is piecewise linear and concave, with a convex abort region that, in spherical coordinates, reduces to radius-based control limits that depend on the angle. A modified PBVI algorithm exploiting these structures enables real-time-ish computation, and extensions to multi-task missions and UAV case studies demonstrate significant cost savings over rule-based and simple CTMC benchmarks. The framework provides asymptotically optimal abort policies for non-Markovian systems and offers a practical approach to robust, data-driven mission abort decisions. The results have direct implications for safety-critical autonomous systems and CBM-like settings where non-Markovian dynamics and imperfect sensing are prevalent.

Abstract

While most on-demand mission-critical systems are engineered to be reliable to support critical tasks, occasional failures may still occur during missions. To increase system survivability, a common practice is to abort the mission before an imminent failure. We consider optimal mission abort for a system whose deterioration follows a general three-state (normal, defective, failed) semi-Markov chain. The failure is assumed self-revealed, while the healthy and defective states have to be {inferred} from imperfect condition monitoring data. Due to the non-Markovian process dynamics, optimal mission abort for this partially observable system is an intractable stopping problem. For a tractable solution, we introduce a novel tool of Erlang mixtures to approximate non-exponential sojourn times in the semi-Markov chain. This allows us to approximate the original process by a surrogate continuous-time Markov chain whose optimal control policy can be solved through a partially observable Markov decision process (POMDP). We show that the POMDP optimal policies converge almost surely to the optimal abort decision rules when the Erlang rate parameter diverges. This implies that the expected cost by adopting the POMDP solution converges to the optimal expected cost. Next, we provide comprehensive structural results on the optimal policy of the surrogate POMDP. Based on the results, we develop a modified point-based value iteration algorithm to numerically solve the surrogate POMDP. We further consider mission abort in a multi-task setting where a system executes several tasks consecutively before a thorough inspection. Through a case study on an unmanned aerial vehicle, we demonstrate the capability of real-time implementation of our model, even when the condition-monitoring signals are generated with high frequency.
Paper Structure (44 sections, 19 theorems, 82 equations, 4 figures, 13 tables, 3 algorithms)

This paper contains 44 sections, 19 theorems, 82 equations, 4 figures, 13 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Problem Statement
  4. System Failure Process
  5. Observed Signals
  6. Mission Abort Decision Making
  7. POMDP Formulation
  8. Erlang Approximation
  9. Bellman Recursion
  10. Structural Properties
  11. Computational Solution Approach
  12. Insights into Specialized Models
  13. Markovian Failure Process
  14. One-Phase Transition
  15. Extension to Multiple Tasks
  16. ...and 29 more sections

Key Result

Proposition 1

Consider the CTMC ${X}^{(\lambda)}(\cdot)$ with initial probabilities in eq:init_prob and transition rate matrix $\mathbf{Q}$ in eq:trans_prob. Then the first hitting time of ${X}^{(\lambda)}(\cdot)$ to the states in $\mathcal{X}_2$ has the same distribution as $T_{12}^{(\lambda)}$, and the absorpti

Figures (4)

  • Figure 1: Two CTMC schemes with $m$ transient states and an absorbing state $A$. With proper choice of the initial probabilities $(\pi_i)_{i\in[m]}$ and the transition rates, their absorbing times can follow the same distribution. The two schemes are used to approximate the transitions from $\mathcal{X}_1$ to $\mathcal{X}_2$ and from $\mathcal{X}_2$ to $\mathcal{X}_3$, respectively.
  • Figure 2: Approximating the distribution of $T_{23}$ by the Erlang mixture distribution $F^{(\lambda)}(\cdot)$: The left panel uses $m_2= 20$ and $\lambda=0.134$ to approximate the Weibull distribution in \ref{['eq:weibull']}. The right panel uses $m_2=50$ and $\lambda=0.209$ to approximate the mixture distribution in \ref{['eq:mixture']}.
  • Figure EC.1: Approximating the distribution $F$ of $T_{23}$ in \ref{['eq:weibull']} by the Erlang mixture distribution $F^{(\lambda)}(\cdot)$ under various values of $m_2$.
  • Figure EC.2: Approximating the distribution $F$ of $T_{23}$ in \ref{['eq:mixture']} by the Erlang mixture distribution $F^{(\lambda)}(\cdot)$ under various values of $m_2$.

Theorems & Definitions (19)

  • Proposition 1
  • Theorem 1
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Theorem 2
  • Theorem 3
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • ...and 9 more