Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exact sensitivity analysis of Markov reward processes via algebraic geometry

Timothy C. Y. Chan, Muhammad Maaz

Abstract

We introduce a new approach for deterministic sensitivity analysis of Markov reward processes, commonly used in cost-effectiveness analyses, via reformulation into a polynomial system. Our approach leverages cylindrical algebraic decomposition (CAD), a technique arising from algebraic geometry that provides an exact description of all solutions to a polynomial system. While it is typically intractable to build a CAD for systems with more than a few variables, we show that a special class of polynomial systems, which includes the polynomials arising from Markov reward processes, can be analyzed much more tractably. We establish several theoretical results about such systems and develop a specialized algorithm to construct their CAD, which allows us to perform exact, multi-way sensitivity analysis for common health economic analyses. We develop an open-source software package that implements our algorithm. Finally, we apply it to two case studies, one with synthetic data and one that re-analyzes a previous cost-effectiveness analysis from the literature, demonstrating advantages of our approach over standard techniques. Our software and code are available at: \url{https://github.com/mmaaz-git/markovag}.

Exact sensitivity analysis of Markov reward processes via algebraic geometry

Abstract

We introduce a new approach for deterministic sensitivity analysis of Markov reward processes, commonly used in cost-effectiveness analyses, via reformulation into a polynomial system. Our approach leverages cylindrical algebraic decomposition (CAD), a technique arising from algebraic geometry that provides an exact description of all solutions to a polynomial system. While it is typically intractable to build a CAD for systems with more than a few variables, we show that a special class of polynomial systems, which includes the polynomials arising from Markov reward processes, can be analyzed much more tractably. We establish several theoretical results about such systems and develop a specialized algorithm to construct their CAD, which allows us to perform exact, multi-way sensitivity analysis for common health economic analyses. We develop an open-source software package that implements our algorithm. Finally, we apply it to two case studies, one with synthetic data and one that re-analyzes a previous cost-effectiveness analysis from the literature, demonstrating advantages of our approach over standard techniques. Our software and code are available at: \url{https://github.com/mmaaz-git/markovag}.
Paper Structure (29 sections, 17 theorems, 26 equations, 6 figures, 1 table, 3 algorithms)

This paper contains 29 sections, 17 theorems, 26 equations, 6 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Cost-effectiveness analysis
  4. Uncertainty in Markov models
  5. Cylindrical algebraic decomposition
  6. Semialgebraic Representations of Markov Reward Processes
  7. Total reward.
  8. Net monetary benefit (NMB).
  9. Incremental cost-effectiveness ratio (ICER).
  10. Cylindrical Algebraic Decomposition
  11. The CAD Algorithm
  12. A Special Class of Polynomial System
  13. CAD of simplices.
  14. Simplex-extensibility: Lifting the simplex CAD to include $f^*$.
  15. Solution formula construction.
  16. ...and 14 more sections

Key Result

Lemma 1

$R_\infty$ is a ratio of two polynomials in $\boldsymbol{\mathbf{\pi}}$, $\boldsymbol{\mathbf{P}}$, and $\boldsymbol{\mathbf{r}}$. Furthermore, $\det(\boldsymbol{\mathbf{I}} - \lambda \boldsymbol{\mathbf{P}}) > 0$ and the adjugate $\mathop{\mathrm{adj}}\nolimits(\boldsymbol{\mathbf{I}} - \lambda \bo

Figures (6)

  • Figure 1: Valid parameter space of a $n=3$ state Markov chain with states (1) healthy, (2) sick, (3) dead, as discussed in Section \ref{['sec:synth_casestudy']}, with the parameters $p_{12} = 0.4, p_{21} = 0.1, r_1=1, r_2=0.5$. We assert that $R_\infty \geq 3$. Green and red points correspond to the grid search, where green indicates a valid point and red an invalid one. The green lines form the convex hull of the green points.
  • Figure 2: Visualization of the parameter space over which the drone network is cost-effective. The shaded gray region is the exact analytic solution obtained by markovag. The points represent the traditional mesh grid approach, with green representing a valid point and red invalid.
  • Figure :
  • Figure :
  • Figure :
  • ...and 1 more figures

Theorems & Definitions (42)

  • Lemma 1
  • Example 1
  • Remark 1: Death state
  • Definition 1: Cell
  • Example 2
  • Example 3
  • Theorem 1
  • Remark 2
  • Corollary 1
  • Example 4
  • ...and 32 more