Enhancing Study-Level Inference from Clinical Trial Papers via Reinforcement Learning-Based Numeric Reasoning

TL;DR

This work reframes systematic-review automation as a quantitative reasoning problem, moving beyond retrieval-based textual inference to structured numeric interpretation of study data. It introduces a two-component pipeline: a numeric data extraction model (trained via supervised fine-tuning and reinforcement learning) and an effect-estimate module that computes standard measures and derives study conclusions. A novel RL approach with fine-grained rewards (CR, FR, TFR) and a GRPO objective yields substantial gains, achieving up to 21% absolute improvement in F1 on CochraneForest and surpassing large-scale LLM baselines on RCTs by up to 9%. The results demonstrate that reasoning-driven supervision over numeric content can deliver interpretable, domain-aligned automation for evidence synthesis, with potential to streamline forest-plot generation and downstream meta-analysis.

Abstract

Systematic reviews in medicine play a critical role in evidence-based decision-making by aggregating findings from multiple studies. A central bottleneck in automating this process is extracting numeric evidence and determining study-level conclusions for specific outcomes and comparisons. Prior work has framed this problem as a textual inference task by retrieving relevant content fragments and inferring conclusions from them. However, such approaches often rely on shallow textual cues and fail to capture the underlying numeric reasoning behind expert assessments. In this work, we conceptualise the problem as one of quantitative reasoning. Rather than inferring conclusions from surface text, we extract structured numerical evidence (e.g., event counts or standard deviations) and apply domain knowledge informed logic to derive outcome-specific conclusions. We develop a numeric reasoning system composed of a numeric data extraction model and an effect estimate component, enabling more accurate and interpretable inference aligned with the domain expert principles. We train the numeric data extraction model using different strategies, including supervised fine-tuning (SFT) and reinforcement learning (RL) with a new value reward model. When evaluated on the CochraneForest benchmark, our best-performing approach -- using RL to train a small-scale number extraction model -- yields up to a 21% absolute improvement in F1 score over retrieval-based systems and outperforms general-purpose LLMs of over 400B parameters by up to 9% on the RCTs benchmark. Our results demonstrate the promise of reasoning-driven approaches for automating systematic evidence synthesis.

Figures (9)

• Figure 1: Example of estimating the intervention effect based on the extracted outcome data for a clinical study.
• Figure 2: F1 score for predicting the correct answers on 30 instances from CochraneForest on different retrieval precision using 4 state-of-the-art LLMs.
• Figure 3: Rewards dynamics. Format rewards plateau quickly during training, whereas the correctness reward improves more gradually, suggesting that LLMs naturally learn to follow structural patterns while still improving on quality rewards.
• Figure 4: A forest plot assessing clinical remission (binary outcome) in patients affected by medically refractory Crohn’s disease treated with stem cells versus control.
• Figure 5: A forest plot comparing GnRH agonist versus no agonist (placebo) in women with endometriosis, assessing a continuous outcome (average number of oocytes per woman).