SECRET: Semi-supervised Clinical Trial Document Similarity Search

TL;DR

SECRET tackles the challenge of identifying similar clinical trial protocols by framing trials as sets of question-and-answer pairs and training with both local (Q/A-level) and global (trial-level) contrastive objectives. This semi-supervised approach reduces reliance on large labeled datasets and addresses long-document issues by summarizing content into Q/A pairs, preserving critical information. Empirical results show SECRET outperforms strong baselines, including Trial2Vec, across complete, partial, and zero-shot patient-to-trial matching tasks, often with substantial gains in recall and precision. The method offers a practical, scalable path for faster and more accurate trial design assistance in real-world settings.

Abstract

Clinical trials are vital for evaluation of safety and efficacy of new treatments. However, clinical trials are resource-intensive, time-consuming and expensive to conduct, where errors in trial design, reduced efficacy, and safety events can result in significant delays, financial losses, and damage to reputation. These risks underline the importance of informed and strategic decisions in trial design to mitigate these risks and improve the chances of a successful trial. Identifying similar historical trials is critical as these trials can provide an important reference for potential pitfalls and challenges including serious adverse events, dosage inaccuracies, recruitment difficulties, patient adherence issues, etc. Addressing these challenges in trial design can lead to development of more effective study protocols with optimized patient safety and trial efficiency. In this paper, we present a novel method to identify similar historical trials by summarizing clinical trial protocols and searching for similar trials based on a query trial's protocol. Our approach significantly outperforms all baselines, achieving up to a 78% improvement in recall@1 and a 53% improvement in precision@1 over the best baseline. We also show that our method outperforms all other baselines in partial trial similarity search and zero-shot patient-trial matching, highlighting its superior utility in these tasks.

Figures (7)

• Figure 1: Overview of SECRET. SECRET consists of three main components: Q/A Generation, Local Contrastive Learning and Global Contrastive Learning. Without loss of generality, we illustrate Sec 1 in the Local Contrastive Learning block, applicable to all sections.
• Figure 2: Ablation results
• Figure 3: Distribution of Q/A Count per Trial via LLM
• Figure 4: Effect of batch size on validation scores.
• Figure 5: Performance of SECRET on the partial retrieval scenarios. We use different sections with title of the trial as queries to retrieve similar trials, including keyword kw, intervention int, disease dz, outcome out, eligibility criteria ec.