Towards Regulatory-Confirmed Adaptive Clinical Trials: Machine Learning Opportunities and Solutions

TL;DR

This work tackles the gap between regulatory-approved clinical trials and real-world treatment policy performance by introducing PTMB and PTF as forward-looking objectives. It proposes RFAN, a two-stage framework combining a regulatory-compliant randomized stage with an adaptive augmentation stage guided by causal-BALD-inspired acquisition and interim switching via alpha-spending. The authors formalize the problem, define policy-valued objectives, and demonstrate that RFAN can yield superior post-trial policy values and fairness (especially for under-served subgroups) without compromising the trial's regulatory guarantees. Through synthetic and semi-synthetic experiments on warfarin and COVID-19 datasets, RFAN shows meaningful improvements in subgroup fairness and treatment policy utility, highlighting its potential to inform post-deployment decision-making and resource allocation.

Abstract

Randomized Controlled Trials (RCTs) are the gold standard for evaluating the effect of new medical treatments. Treatments must pass stringent regulatory conditions in order to be approved for widespread use, yet even after the regulatory barriers are crossed, real-world challenges might arise: Who should get the treatment? What is its true clinical utility? Are there discrepancies in the treatment effectiveness across diverse and under-served populations? We introduce two new objectives for future clinical trials that integrate regulatory constraints and treatment policy value for both the entire population and under-served populations, thus answering some of the questions above in advance. Designed to meet these objectives, we formulate Randomize First Augment Next (RFAN), a new framework for designing Phase III clinical trials. Our framework consists of a standard randomized component followed by an adaptive one, jointly meant to efficiently and safely acquire and assign patients into treatment arms during the trial. Then, we propose strategies for implementing RFAN based on causal, deep Bayesian active learning. Finally, we empirically evaluate the performance of our framework using synthetic and real-world semi-synthetic datasets.

Figures (6)

• Figure 1: Two Phase III trials with different sample distributions, their impact on the resulting treatment effect estimates (top), and on treatment policy at Phase IV (bottom). Left: Confident treatment effect estimates for the majority (M) subgroup with high uncertainty for the minority groups (m), leading to uncertain treatment policy at Phase IV. Right: While the majority group still can safely benefit from the treatment, better treatment estimates for minority groups are estimated, leading to a beneficial treatment policy.
• Figure 2: RFAN timeline. $t^*$ and $\alpha'$ are free parameters. The randomized stage establishes a standard regulatory objective ($\eta$), and the augmented stage actively refines a treatment policy $\pi$, jointly addressing our objectives.
• Figure 3: Distribution of actively acquired synthetic trial population and resulting CATE function.
• Figure 4: Performance over varying sample sizes ($N$) on the synthetic data, over $40$ random seeds
• Figure A.1: Performance over varying $t*$ on synthetic data (N=$300$, T=$30$)