An information metric for comparing and assessing informative interim decisions in sequential clinical trials

TL;DR

This work addresses bias in Bayesian treatment-effect inference arising from informative interim decisions in group-sequential trials. It introduces adaptation-induced posterior divergence (AIPD), a KL-divergence–based metric that quantifies how conditioning on interim decisions distorts posterior beliefs relative to a fixed-sample design, and provides a pre-experimental version for boundary planning. Through normal-data illustrations and a CNS trial example, the authors show that adaptive decisions can widen credible intervals and shift posteriors, and they demonstrate how AIPD can guide design choices by evaluating trade-offs between efficiency and information loss. The approach offers a practical framework for post-hoc evaluation and pre-experimental design of adaptive trials, complementing traditional type-I error considerations and enabling more informed planning of future studies.

Abstract

Group sequential designs enable interim analyses and potential early stopping for efficacy or futility. While these adaptations improve trial efficiency and ethical considerations, they also introduce bias into the adapted analyses. We demonstrate how failing to account for informative interim decisions in the analysis can substantially affect posterior estimates of the treatment effect, often resulting in overly optimistic credible intervals aligned with the stopping decision. Drawing on information theory, we use the Kullback-Leibler divergence to quantify this distortion and highlight its use for post-hoc evaluation of informative interim decisions, with a focus on end-of-study inference. Unlike pointwise comparisons, this measure provides an integrated summary of this distortion on the whole parameter space. By comparing alternative decision boundaries and prior specifications, we illustrate how this measure can improve the understanding of trial results and inform the planning of future adaptive studies. We also introduce an expected version of this metric to support clinicians in choosing decision boundaries. This guidance complements traditional strategies based on type-I error rate control by offering insights into the distortion introduced to the treatment effect at each interim phase. The use of this pre-experimental measure is finally illustrated in a group sequential trial for evaluating a treatment for central nervous system disorders.

Figures (5)

• Figure 1: Unconditional (solid line) and conditional (dashed line) posterior distributions computed at interim and final analyses for different values of $\bar{x}_{(s)}$. Data collected at second interim and final analyses are conditional on having continued the trial up to that point. The vector of futility boundaries is $(-0.85, -0.43, -0.28)$, while efficacy boundaries are $(0.85, 0.43, 0.28)$. Prior distribution (dotted line) used in the analysis phase is $N(0,1.67^2)$ and it is the same for all stages and scenarios.
• Figure 2: Unconditional (solid line) and conditional (dashed line) posterior distributions when trial is stopped for efficacy at the first interim for different values of $\bar{x}_{(1)}$ and efficacy boundary $e_1=0.85$. Prior distribution (dotted line) used in the analysis phase is $N(0,1.67^2)$ and it is the same for all stages and scenarios.
• Figure 3: AIPD as function of the observed sample mean, $\bar{x}_{(s)}$, at interim and final analyses of a $3$-stage group sequential design. Vertical lines show informative efficacy (solid line) and futility (dashed line) boundaries at the two interim analyses. Different normal priors have been chosen: centered on a null treatment effect and either weakly-informative (points) or moderately-informative (triangles); moderately-informative and centered around a positive treatment effect (squares).
• Figure 4: AIPD at interim 2 of a group sequential design with more than two stages for different combinations of the first two efficacy boundaries ($e_2\leq e_1$), given (a) $\bar{x}_{(2)}=0.3$ and (b) $\bar{x}_{(2)}=0.6$. Futility boundaries at interim 1 and 2 are $f_1=-0.85$ and $f_2=-0.43$. Each stage is composed by 12 patients and the prior distribution used in the analysis phase is $N(0,1.67^2)$.
• Figure 5: Expected sample size (top panels) and expected end-of-study adaptation-induced posterior divergence (bottom panels), $\bar{D}(\theta)$, when using Pocock-type and O'Brien-Fleming-type efficacy boundaries in a 3-stage trial with normal endpoint. Each column considers different schedules of interim analyses. All boundary options guarantee a maximum type-I error probability $\alpha=0.025$ under $H_0:\delta\leq0$ (solid line) and power $1-\beta=0.9$ under $H_1:\delta=0.265$ (dashed line). Maximum sample size ranges from 167 to 176 for Pocock-type and 151 to 153 for O’Brien-Fleming-type boundaries across the three interim schedules.