Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing Long-Form Clinical Text Generation with Claim-Based Rewards

Samyak Jhaveri, Praphul Singh, Jangwon Kim, Tara Taghavi, Krishnaram Kenthapadi

TL;DR

This paper tackles the challenge of producing accurate, complete long-form clinical notes with large language models by introducing an evaluation-integrated reinforcement learning framework that combines Group Relative Policy Optimization (GRPO) with DocLens, a claim-level evaluator. By deriving deterministic rewards from claim recall and precision, the method directly optimizes factual grounding and completeness without a separate reward model or ground-truth targets. Empirical results on two clinical benchmarks show improved DocLens precision, recall, and F1, with reward gating accelerating convergence; an independent GPT-5 evaluation corroborates gains in factuality, completeness, and brevity. The approach is scalable and adaptable, capable of incorporating site-specific objectives such as guideline adherence or billing requirements, and represents a practical, cost-efficient path for deploying clinically grounded long-form text generation.

Abstract

Automating clinical documentation with large language models requires precise alignment with priorities such as completeness and factual grounding. We present an evaluation-integrated reinforcement learning framework for long-form clinical text generation that couples Group Relative Policy Optimization (GRPO) with DocLens, a claim-level evaluator that provides deterministic, dialogue-grounded rewards. Our method directly optimizes factual grounding and completeness without training a separate reward model or relying on human-authored references. Empirically, the approach improves clinical note quality and reduces training cost via a simple reward-gating strategy. An independent GPT-5 qualitative evaluation further supports these gains, showing higher preference for GRPO outputs in factuality, completeness, and brevity, with fewer omissions and hallucinations. Because the benchmarks are relatively clean and the base model already well aligned, these improvements likely represent a conservative lower bound. The framework is scalable to real-world settings and can incorporate custom objectives such as guideline adherence or billing preferences.

Optimizing Long-Form Clinical Text Generation with Claim-Based Rewards

TL;DR

This paper tackles the challenge of producing accurate, complete long-form clinical notes with large language models by introducing an evaluation-integrated reinforcement learning framework that combines Group Relative Policy Optimization (GRPO) with DocLens, a claim-level evaluator. By deriving deterministic rewards from claim recall and precision, the method directly optimizes factual grounding and completeness without a separate reward model or ground-truth targets. Empirical results on two clinical benchmarks show improved DocLens precision, recall, and F1, with reward gating accelerating convergence; an independent GPT-5 evaluation corroborates gains in factuality, completeness, and brevity. The approach is scalable and adaptable, capable of incorporating site-specific objectives such as guideline adherence or billing requirements, and represents a practical, cost-efficient path for deploying clinically grounded long-form text generation.

Abstract

Automating clinical documentation with large language models requires precise alignment with priorities such as completeness and factual grounding. We present an evaluation-integrated reinforcement learning framework for long-form clinical text generation that couples Group Relative Policy Optimization (GRPO) with DocLens, a claim-level evaluator that provides deterministic, dialogue-grounded rewards. Our method directly optimizes factual grounding and completeness without training a separate reward model or relying on human-authored references. Empirically, the approach improves clinical note quality and reduces training cost via a simple reward-gating strategy. An independent GPT-5 qualitative evaluation further supports these gains, showing higher preference for GRPO outputs in factuality, completeness, and brevity, with fewer omissions and hallucinations. Because the benchmarks are relatively clean and the base model already well aligned, these improvements likely represent a conservative lower bound. The framework is scalable to real-world settings and can incorporate custom objectives such as guideline adherence or billing preferences.

Paper Structure

This paper contains 26 sections, 6 equations, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Clinical Text Generation
  4. Reinforcement Learning for Clinical Text Alignment
  5. Evaluation-Integrated Reward Modeling
  6. Method
  7. Pipeline Overview
  8. Reward Computation
  9. Reward Gating
  10. Group Relative Policy Optimization
  11. Training Algorithm
  12. Experimental Setup
  13. Datasets
  14. Models
  15. Training
  16. ...and 11 more sections

Figures (3)

  • Figure 1: (Top) Reference claims extraction from doctor-patient dialogue, pre-computed using GPT-4o and cached for reuse across training iterations. (Bottom) GRPO training pipeline. Group of SOAP notes generated for each doctor-patient dialogue, evaluated for completeness (claim recall) and factual grounding (claim precision) in comparison to dialogue-grounded reference claims, yielding a reward signal to update the policy model via RL.
  • Figure 2: DocLens metrics illustrated: Completeness (Claim Recall) measures the proportion of claims from the source conversation that can be entailed by the generated SOAP note. Factual Grounding (Claim Precision) measures the proportion of claims from the generated SOAP note that can be entailed by the source conversation.
  • Figure 3: Qualitative evaluation on ACI-Bench using GPT-5 as an external judge. Top: pairwise preference outcomes for factuality, completeness, organization, and brevity. Bottom-left: total omissions by SOAP section. Bottom-right: smoothed CDF of hallucination counts per note.