Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Q-approximation of operating characteristics of clinical trial designs

Susanna Gentile, Daniel E. Schwartz, Riddhiman Saha, Lorenzo Trippa

Abstract

Designing clinical trials requires evaluating multiple operating characteristics (OCs), such as the likelihood of an early stopping decision, the probability of detecting a treatment effect, and the Type I error rate. In most cases, these evaluations are based on computationally intensive Monte Carlo simulations. As the complexity of clinical trials and the use of adaptive designs increase, the computational burden can quickly become prohibitive. We introduce a strategy for rapidly approximating OCs, called the Q-approximation. Our approach is based on quadratic approximations of the log-likelihood and asymptotic arguments. The Q-approximation approach can be applied to any trial design that uses data analysis methods coherent with the likelihood principle, including multistage designs with early stopping, adaptively randomized designs, and designs that leverage external data. We illustrate the approach with several examples and show that it provides an accurate approximation of important OCs while reducing the computation time compared to Monte Carlo simulations. In particular, in our experiments, the standard Monte Carlo approximation of OCs requires 150 to 1,900 times greater computing budget than Q-approximations to achieve comparable levels of accuracy.

Q-approximation of operating characteristics of clinical trial designs

Abstract

Designing clinical trials requires evaluating multiple operating characteristics (OCs), such as the likelihood of an early stopping decision, the probability of detecting a treatment effect, and the Type I error rate. In most cases, these evaluations are based on computationally intensive Monte Carlo simulations. As the complexity of clinical trials and the use of adaptive designs increase, the computational burden can quickly become prohibitive. We introduce a strategy for rapidly approximating OCs, called the Q-approximation. Our approach is based on quadratic approximations of the log-likelihood and asymptotic arguments. The Q-approximation approach can be applied to any trial design that uses data analysis methods coherent with the likelihood principle, including multistage designs with early stopping, adaptively randomized designs, and designs that leverage external data. We illustrate the approach with several examples and show that it provides an accurate approximation of important OCs while reducing the computation time compared to Monte Carlo simulations. In particular, in our experiments, the standard Monte Carlo approximation of OCs requires 150 to 1,900 times greater computing budget than Q-approximations to achieve comparable levels of accuracy.
Paper Structure (16 sections, 2 theorems, 24 equations, 1 figure, 2 tables, 8 algorithms)

This paper contains 16 sections, 2 theorems, 24 equations, 1 figure, 2 tables, 8 algorithms.

Table of Contents

  1. Introduction
  2. Method
  3. Notation
  4. Q-approximation of OCs
  5. Graphing the key steps of the Q-approximation procedure
  6. Evaluating the accuracy of Q-approximations
  7. Q-approximations with multi-stage designs
  8. An RCT design that leverages external data
  9. Trial protocol and simulation scenarios
  10. Algorithms to approximate the OCs
  11. OC estimates
  12. A multi-arm RCT with Bayesian adaptive randomization
  13. Trial protocol and simulation scenarios
  14. Algorithms to approximate operating characteristics
  15. A comparison of MC-approximations and Q-approximations
  16. ...and 1 more sections

Key Result

Proposition 1

In Example 2, as $R \to \infty$, we have $\hat{\psi}_{Q,n} \overset{p}{\to} \psi_{Q,n} = E_{f_{\omega_n, \mathcal{T}}} \left[ h' \left( \widetilde{L} \left( \cdot; C^{(r)}, V \right) \right) \right]$. Also, as $n \to \infty$, we have $\psi_{Q,n} - \psi_n \to 0$.

Figures (1)

  • Figure 1: Summary of the Q-approximation for Example 2, $n = 100$, $n_0 = n_1 = 50$, and $\omega_{100} = (0.4, 0.61)$. Panel A shows, for a single dataset, the similarity between the likelihood $L(\theta; D)$ and its Gaussian approximation $\widetilde{L} \left( \theta; \hat{\theta}, J \right)$ from equation \ref{['eq:gaussian_lik_ex2']}. For the control group, the blue curve is $L \left( (\cdot, \hat{\theta}_1); D \right)$, a function of $\theta_0$, and the red curve is the approximation $\widetilde{L} \left( ( \cdot,\hat{\theta}_1); \hat{\theta}, J \right)$. Panel B compares the distribution of the MLE $(\hat{\theta}_0, \hat{\theta}_1)$ across 1,000 datasets $D^{(r)} \overset{iid}{\sim} g_{\omega_{100}, \mathcal{T}}$ and the asymptotic approximation $f_{\omega_{100}, \mathcal{T}}$ used to sample $C^{(r)}$ (alg \ref{['alg:Q_ex2']}, line 2). Panel C shows the sampling distribution of the ratio between the observed information $J_{0,0}$ (from equation \ref{['eq:gaussian_lik_ex2']}) and the fixed curvature $V_{0,0} = n_0 \mathcal{J}_{0,0} = \frac{n_0}{\omega_0 ( 1 - \omega_0 )}$ (the first entry of the $V$ matrix in Algorithm \ref{['alg:Q_ex2']}, line 2), as $n_0$ increases. Bars show the 5% and 95% quantiles. Panel D includes four replicates of the function $h'\left( \widetilde{L} \left( \cdot; C^{(r,P)}, V^{(P)} \right) \right) = \mathbbm{1}\left\{\widetilde{\Pi}_\delta \left( (0, \infty]; C_\delta^{(r,P)}, V_\delta^{(P)} \right) > 0.9 \right\}$ (alg \ref{['alg:Q_ex2']}, line 3). Panel E illustrates results of the Q-approximation (red, Algorithm \ref{['alg:Q_ex2']}) and the MC-approximation (blue, Algorithm \ref{['alg:MC_ex2']}) for the sequence of scenarios $\omega_n = (0.4, 0.4 + \frac{2.1}{\sqrt{n}})$, with $R = 100,000$ and $M = 20,000$ draws from the posterior in Algorithm \ref{['alg:MC_ex2']}. Panel F compares the RMSE of $\hat{\psi}_{MC}$ (Algorithm \ref{['alg:MC_ex2']}, with M = 20,000) with $\hat{\psi}_{Q}$ (Algorithm \ref{['alg:Q_ex2']}). It shows the decrease of the RMSE as a function of the time budget (in seconds). Approximations used as many replicates as possible for a given time budget. Since one MC replicate requires approximately 2.1 milliseconds, the RMSE for the MC estimator cannot be computed for the smallest time budgets.

Theorems & Definitions (2)

  • Proposition 1
  • Corollary 1