Constrained Markov decision processes for response-adaptive procedures in clinical trials with binary outcomes

TL;DR

The CMDP approach provides a previously unexplored way to construct response-adaptive procedures with a desired degree of robustness to prior misspecification in terms of expected treatment outcomes and is presented as a feasible policy accompanied by a respective optimality gap.

Abstract

A constrained Markov decision process (CMDP) approach is developed for response-adaptive procedures in clinical trials with binary outcomes. The resulting CMDP class of Bayesian response -- adaptive procedures can be used to target a certain objective, e.g., patient benefit or power while using constraints to keep other operating characteristics under control. In the CMDP approach, the constraints can be formulated under different priors, which can induce a certain behaviour of the policy under a given statistical hypothesis, or given that the parameters lie in a specific part of the parameter space. A solution method is developed to find the optimal policy, as well as a more efficient method, based on backward recursion, which often yields a near-optimal solution with an available optimality gap. Three applications are considered, involving type I error and power constraints, constraints on the mean squared error, and a constraint on prior robustness. While the CMDP approach slightly outperforms the constrained randomized dynamic programming (CRDP) procedure known from literature when focussing on type I and II error and mean squared error, showing the general quality of CRDP, CMDP significantly outperforms CRDP when the focus is on type I and II error only.

Figures (18)

• Figure 1: Patient benefit (subfigure a), rejection rate (subfigure b), bias (subfigure c), and mean squared error (subfigure d) vs. $\theta_\text{\normalfont D}$ for $\theta_\text{\normalfont C}=0.5$, $n=75$, $\alpha^* = 0.05$, $\beta= 0.4$, and RA procedures ER (solid), DP (dashed), CRDP (dotted) and CMDP-T (dash-dotted)
• Figure 2: Patient benefit (subfigure a), rejection rate (subfigure b), bias (subfigure c), and mean squared error (subfigure d) vs. $\theta_\text{\normalfont D}$ for $\theta_\text{\normalfont C}=0.5$, $n=200$, $\alpha^* = 0.07$, $\beta= 0.23$, and RA procedures ER (solid), DP (dashed), CRDP (dotted) and CMDP-T (dash-dotted)
• Figure 3: Patient benefit (subfigure a), rejection rate (subfigure b), bias (subfigure c), and mean squared error (subfigure d) vs. $\theta_\text{\normalfont D}$ for $\theta_\text{\normalfont C}=0.5$, $n=75$, and RA procedures ER (solid), CMDP-E2 (dashed), CRDP (dotted) and CMDP-E1 (dash-dotted)
• Figure 4: Patient benefit (subfigure a), rejection rate (subfigure b), bias (subfigure c), and mean squared error (subfigure d) vs. $\theta_\text{\normalfont D}$ for $\theta_\text{\normalfont C}=0.5$, $n=200$, and RA procedures ER (solid), CMDP-E2 (dashed), CRDP (dotted) and CMDP-E1 (dash-dotted)
• Figure 5: Patient benefit (subfigure a), rejection rate (subfigure b), bias (subfigure c), and mean squared error (subfigure d) vs. $\theta_\text{\normalfont D}$ for $\theta_\text{\normalfont C}=0.3$, $n=200$, $\tilde{s}_{1,0}=3$, $\tilde{f}_{1,0}=7$, $\tilde{s}_{2,0}=6$, $\tilde{f}_{2,0}=4$ and the CMDP-R procedure \ref{['CMDP_ROBUST']} with $\xi=0.00,0.990,0.999,1.00$ denoted by the solid, dashed, dotted and dash-dotted lines respectively.

Theorems & Definitions (12)

• Example 1: Control of power and type I error
• Example 2: Control of estimation error
• Example 3: Robustness to prior misspecification
• Lemma 4
• proof
• Theorem 5
• proof
• Theorem 6
• Theorem 7
• proof