VERI-DPO: Evidence-Aware Alignment for Clinical Summarization via Claim Verification and Direct Preference Optimization

TL;DR

VERI-DPO is introduced, which uses claim verification to mine preferences and distill them into the summarizer with Direct Preference Optimization (DPO), and verifier-mined preferences separate candidates by contradiction density.

Abstract

Brief Hospital Course (BHC) narratives must be clinically useful yet faithful to fragmented EHR evidence. LLM-based clinical summarizers still introduce unsupported statements, and alignment can encourage omissions ("say-less" degeneration). We introduce VERI-DPO, which uses claim verification to mine preferences and distill them into the summarizer with Direct Preference Optimization (DPO). On MIMIC-III-Ext-VeriFact-BHC (100 ICU patients; patient-level splits), we train a retrieval-augmented verifier to label claim-evidence pairs as Supported, Not Supported, or Not Addressed via a single-token format. The verifier scores sentence-level claims from sampled BHC candidates and aggregates margins into a coverage-aware utility to mine length-controlled, contradiction-anchored preference pairs. On held-out patients, verifier-mined preferences separate candidates by contradiction density, and VERI-DPO reduces Not Supported claim rates from 10.7% to 1.9% (local verifier judge) and from 11.6% to 6.4% (GPT-4o judge), while improving validity from 76.7% to 82.5% and maintaining informative length.

Figures (1)

• Figure 1: Overview of VERI-DPO. (A) Verifier training: a compact retrieval-augmented verifier predicts single-token labels A/B/C for each (evidence, claim) pair: A=Supported, B=Not Supported (error/hallucination), and C=Not Addressed (omission / not in retrieved evidence). (B) Verifier-driven preference mining: for each sampled BHC candidate, we decompose it into sentence-level claims, retrieve per-claim evidence, and score claims with the verifier; we aggregate claim-level signals into a utility $U(y)$ that penalizes contradictions (NS/HCNS) while discouraging degeneration-by-omission, and select $(y^{+},y^{-})$ under utility and length/coverage constraints (HCNS: B under $\mathrm{bias}_{\mathrm{prec}}$ with margin $\Delta>\delta$). (C) DPO alignment: mined pairs are used to optimize a single-sample policy that internalizes verifier preferences, avoiding inference-time reranking.