$\texttt{AMEND++}$: Benchmarking Eligibility Criteria Amendments in Clinical Trials

TL;DR

The paper tackles predicting whether eligibility criteria in initial clinical trial protocols will be amended, addressing delays and cost burdens in trial design. It introduces AMEND++, a two-dataset benchmark (AMEND and AMEND_LLM) built from ClinicalTrials.gov EC version histories, and a revision-aware pretraining method CAMLM that leverages historical EC edits. CAMLM consistently improves amendment prediction across multiple backbones and classifiers, with gains validated through ablations and statistically significant results. The work provides a scalable framework for proactive trial design by forecasting amendment risk, while also releasing high-quality labeled data and outlining directions for extending the approach to other protocol sections.

Abstract

Clinical trial amendments frequently introduce delays, increased costs, and administrative burden, with eligibility criteria being the most commonly amended component. We introduce \textit{eligibility criteria amendment prediction}, a novel NLP task that aims to forecast whether the eligibility criteria of an initial trial protocol will undergo future amendments. To support this task, we release $\texttt{AMEND++}$, a benchmark suite comprising two datasets: $\texttt{AMEND}$, which captures eligibility-criteria version histories and amendment labels from public clinical trials, and $\verb|AMEND_LLM|$, a refined subset curated using an LLM-based denoising pipeline to isolate substantive changes. We further propose $\textit{Change-Aware Masked Language Modeling}$ (CAMLM), a revision-aware pretraining strategy that leverages historical edits to learn amendment-sensitive representations. Experiments across diverse baselines show that CAMLM consistently improves amendment prediction, enabling more robust and cost-effective clinical trial design.

Figures (6)

• Figure 1: Overview of the proposed framework for clinical trial eligibility criteria (EC) amendment prediction, from data collection to downstream task application. LR: logistic regression, RF: random forest, TH: trial head used for predicting amendment. Human annotation is performed only on the test data.
• Figure 2: Clinical trial eligibility criteria amendment rates across phases and trial status in the AMEND_LLM dataset (n=64,641) dataset. Dashed lines indicate mean amendment rates for each dimension.
• Figure 3: Clinical trial eligibility criteria amendment rates across phases and trial status in the AMEND (n=161,970) dataset. Dashed lines indicate mean amendment rates for each dimension.
• Figure 4: Prompt given to the LLM for the "Criteria Removed" task.
• Figure 5: Prompt given to the LLM for the "Criteria Modified" task.