Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

POET: Protocol Optimization via Eligibility Tuning

Trisha Das, Katherine Kero, Dorinda Schumann, Tracy Ohrt, Sanjit Singh Batra, Gregory D Lyng, Robert E. Tillman

TL;DR

POET tackles the bottleneck in clinical trial design by enabling interpretable eligibility criteria (EC) generation through semantic axes guided by large language models. It frames EC generation as a masked, axis-constrained prediction problem, complemented by a data-driven masking strategy and a rubric-based evaluation that includes LLM-based judgments aligned with clinicians. Empirical results show guided generation consistently outperforms unguided generation across rarity levels and correlates with clinician assessments, supporting scalable, interpretable AI-assisted trial design. The framework demonstrates a practical path toward reducing clinician burden while improving EC quality and consistency in real-world trial design workflows.

Abstract

Eligibility criteria (EC) are essential for clinical trial design, yet drafting them remains a time-intensive and cognitively demanding task for clinicians. Existing automated approaches often fall at two extremes either requiring highly structured inputs, such as predefined entities to generate specific criteria, or relying on end-to-end systems that produce full eligibility criteria from minimal input such as trial descriptions limiting their practical utility. In this work, we propose a guided generation framework that introduces interpretable semantic axes, such as Demographics, Laboratory Parameters, and Behavioral Factors, to steer EC generation. These axes, derived using large language models, offer a middle ground between specificity and usability, enabling clinicians to guide generation without specifying exact entities. In addition, we present a reusable rubric-based evaluation framework that assesses generated criteria along clinically meaningful dimensions. Our results show that our guided generation approach consistently outperforms unguided generation in both automatic, rubric-based and clinician evaluations, offering a practical and interpretable solution for AI-assisted trial design.

POET: Protocol Optimization via Eligibility Tuning

TL;DR

POET tackles the bottleneck in clinical trial design by enabling interpretable eligibility criteria (EC) generation through semantic axes guided by large language models. It frames EC generation as a masked, axis-constrained prediction problem, complemented by a data-driven masking strategy and a rubric-based evaluation that includes LLM-based judgments aligned with clinicians. Empirical results show guided generation consistently outperforms unguided generation across rarity levels and correlates with clinician assessments, supporting scalable, interpretable AI-assisted trial design. The framework demonstrates a practical path toward reducing clinician burden while improving EC quality and consistency in real-world trial design workflows.

Abstract

Eligibility criteria (EC) are essential for clinical trial design, yet drafting them remains a time-intensive and cognitively demanding task for clinicians. Existing automated approaches often fall at two extremes either requiring highly structured inputs, such as predefined entities to generate specific criteria, or relying on end-to-end systems that produce full eligibility criteria from minimal input such as trial descriptions limiting their practical utility. In this work, we propose a guided generation framework that introduces interpretable semantic axes, such as Demographics, Laboratory Parameters, and Behavioral Factors, to steer EC generation. These axes, derived using large language models, offer a middle ground between specificity and usability, enabling clinicians to guide generation without specifying exact entities. In addition, we present a reusable rubric-based evaluation framework that assesses generated criteria along clinically meaningful dimensions. Our results show that our guided generation approach consistently outperforms unguided generation in both automatic, rubric-based and clinician evaluations, offering a practical and interpretable solution for AI-assisted trial design.
Paper Structure (20 sections, 6 equations, 13 figures, 5 tables)

This paper contains 20 sections, 6 equations, 13 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Eligibility Criteria Generation and Recommendation
  4. Method
  5. Problem Formulation
  6. Masking Strategy
  7. Criteria Generation using LLM
  8. Rubric Creation
  9. Experiments and Results
  10. Experimental Setup
  11. Results and Discussion
  12. Experiments on Guided vs Unguided Generation
  13. Experiments of Number of Generation
  14. Clinician Evaluation
  15. Case Studies
  16. ...and 5 more sections

Figures (13)

  • Figure 1: Framework of POET
  • Figure 2: Usage of POET
  • Figure 3: Comparison among different GPT models on rare data. Total score is computed as the sum of three components: Criteria Similarity (normalized between 0 and 1), Axis Similarity, and Rarity Similarity. Values are reported as mean $\pm$ standard deviation.
  • Figure 4: Comparison of guided and unguided approaches across four metrics—Criteria Similarity (range: 0–3), Axis Similarity (binary: 0 or 1), Rarity Similarity (binary: 0 or 1), and BERTScore (continuous, between 0 and 1)—for Best of 1, 5, and 10 generations compared to corresponding target criteria in rare category. Error bars represent standard deviation.
  • Figure 5: Improvement in mean Total Score for guided over unguided generation across rare, medium, and common categories.
  • ...and 8 more figures