Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conservative Software Reliability Assessments Using Collections of Bayesian Inference Problems

Kizito Salako, Rabiu Tsoho Muhammad

TL;DR

This work addresses obtaining worst-case posterior predictive probabilities for software reliability by aggregating over collections of priors (a credal set) in a Bernoulli failure model. It unifies and extends conservative Bayesian inference (*cbi) by showing the problem reduces to a nonlinear fractional program with an explicit fixed-point solution, characterized by a unique triplet \((\phi^*, y_{**}, y_*)\) and an attracting/repelling structure via the function \(h(x)\). The results connect conservatism to practical reliability assessments, demonstrating convergence to traditional Bayesian results as priors become fully specified and highlighting substantial differences in required evidence under *cbi for safety-critical applications. The findings emphasize that conservatism is relative to the available evidence and model assumptions, offer guidance for numerical solution and interpretation, and point to future work in multi-objective, imprecise-prior, and non-iid extensions for broader dependability assessments.

Abstract

When using Bayesian inference to support conservative software reliability assessments, it is useful to consider a collection of Bayesian inference problems, with the aim of determining the worst-case value (from this collection) for a posterior predictive probability that characterizes how reliable the software is. Using a Bernoulli process to model the occurrence of software failures, we explicitly determine (from collections of Bayesian inference problems) worst-case posterior predictive probabilities of the software operating without failure in the future. We deduce asymptotic properties of these conservative posterior probabilities and their priors, and illustrate how to use these results in assessments of safety-critical software. This work extends robust Bayesian inference results and so-called conservative Bayesian inference methods.

Conservative Software Reliability Assessments Using Collections of Bayesian Inference Problems

TL;DR

This work addresses obtaining worst-case posterior predictive probabilities for software reliability by aggregating over collections of priors (a credal set) in a Bernoulli failure model. It unifies and extends conservative Bayesian inference (*cbi) by showing the problem reduces to a nonlinear fractional program with an explicit fixed-point solution, characterized by a unique triplet \((\phi^*, y_{**}, y_*)\) and an attracting/repelling structure via the function \(h(x)\). The results connect conservatism to practical reliability assessments, demonstrating convergence to traditional Bayesian results as priors become fully specified and highlighting substantial differences in required evidence under *cbi for safety-critical applications. The findings emphasize that conservatism is relative to the available evidence and model assumptions, offer guidance for numerical solution and interpretation, and point to future work in multi-objective, imprecise-prior, and non-iid extensions for broader dependability assessments.

Abstract

When using Bayesian inference to support conservative software reliability assessments, it is useful to consider a collection of Bayesian inference problems, with the aim of determining the worst-case value (from this collection) for a posterior predictive probability that characterizes how reliable the software is. Using a Bernoulli process to model the occurrence of software failures, we explicitly determine (from collections of Bayesian inference problems) worst-case posterior predictive probabilities of the software operating without failure in the future. We deduce asymptotic properties of these conservative posterior probabilities and their priors, and illustrate how to use these results in assessments of safety-critical software. This work extends robust Bayesian inference results and so-called conservative Bayesian inference methods.

Paper Structure

This paper contains 13 sections, 4 theorems, 36 equations, 13 figures.

Table of Contents

  1. Introduction
  2. Results
  3. Discussion
  4. Bayesian Inference for Conservative Assessments
  5. Properties of CBI Solutions
  6. Practical Applications and Implications
  7. Limitations and Future Work
  8. Proof of Proposition \ref{['prop_cbi_transform']}
  9. Proof of Theorem \ref{['thm_gen_sol']}
  10. Proof that $\phi^*\to(1-y_2)^m$ monotonically, as $k \to\infty$
  11. The ratio of the number of demands needed to demonstrate a reliability bound --- using a Beta prior vs using a *cbi prior --- is asymptotically bounded.
  12. Convergence properties of $y_{*}$ and $y_{**}$ for fixed $\phi^*$
  13. Proof of the convergence of Corollary \ref{['cor_special_case_no_failure_solution']} to the solutions in strigini_software_2013

Key Result

Proposition 1

For any $\mathbb P\in\mathcal{D}$, $\dfrac{\mathbb E[f(X)]}{\mathbb E[g(X)]}=\frac{\sum\limits_{i=1}^{n}f(x_i)\mathbb P(A_i)}{\sum\limits_{i=1}^{n}g(x_i)\mathbb P(A_i)}$ for some $x_i\in I_i$ ($i=1...n$).

Figures (13)

  • Figure 1: An illustration of conservative prior distribution \ref{['eqn_priorCBIsolBer']}
  • Figure 2: How $\phi^*$, $y_{**}$, and $y_*$, are related by $h(x) = (1 - x)^m\left(\frac{r - x(m + k + r)}{r - x(k + r)}\right)$.
  • Figure 3: Ratio of the number of demands needed to demonstrate $(1-\alpha)$ reliability bound with $y_1=10^{-6}$, $y_2=10^{-4}$.
  • Figure 4: Same parameters as Figure \ref{['fig_comp_1996_paper1']} except $\boldsymbol{y_2 = 2\times10^{-5}}$.
  • Figure 5: Same parameters as Figure \ref{['fig_comp_1996_paper1']} except $\boldsymbol{y_2 = 10^{-5}}$
  • ...and 8 more figures

Theorems & Definitions (8)

  • Proposition 1
  • proof
  • Theorem 1
  • proof
  • Corollary 1
  • proof
  • Lemma 1: convergence of $y_\ast$ and $y_{\ast\ast}$ for fixed $\phi^*$
  • proof