Process Reward Models for Sentence-Level Verification of LVLM Radiology Reports

TL;DR

This work tackles the critical problem of factual hallucinations in LVLM-generated radiology reports by introducing a sentence-level Process Reward Model (PRM) that verifies each generated sentence conditioned on clinical context and prior text. The PRM is trained with weak supervision from RadNLI on MIMIC-CXR data and evaluated against strong baselines, demonstrating superior sentence-level discrimination and robust transfer to unseen generators. Downstream, PRM scores enable effective report rejection and a weighted Best-of-N selection strategy, yielding meaningful improvements in clinical metrics like F1-CheXbert and BERTScore. The results support the feasibility of a lightweight, model-agnostic safety layer for clinical LVLMs, with clear paths for calibration, RL integration, and expansion to additional report sections and contexts.

Abstract

Automating radiology report generation with Large Vision-Language Models (LVLMs) holds great potential, yet these models often produce clinically critical hallucinations, posing serious risks. Existing hallucination detection methods frequently lack the necessary sentence-level granularity or robust generalization across different LVLM generators. We introduce a novel approach: a sentence-level Process Reward Model (PRM) adapted for this vision-language task. Our PRM predicts the factual correctness of each generated sentence, conditioned on clinical context and preceding text. When fine-tuned on MIMIC-CXR with weakly-supervised labels, a lightweight 0.5B-parameter PRM outperforms existing verification techniques, demonstrating, for instance, relative improvements of 7.5% in Matthews Correlation Coefficient and 1.8% in AUROC over strong white-box baselines on outputs from one LVLM. Unlike methods reliant on internal model states, our PRM demonstrates strong generalization to an unseen LVLM. We further show its practical utility: PRM scores effectively filter low-quality reports, improving F1-CheXbert scores by 4.5% (when discarding the worst 10% of reports). Moreover, when guiding a novel weighted best-of-N selection process on the MIMIC-CXR test set, our PRM show relative improvements in clinical metrics of 7.4% for F1-CheXbert and 0.6% for BERTScore. These results demonstrate that a lightweight, context-aware PRM provides a model-agnostic safety layer for clinical LVLMs without access to internal activations

Figures (14)

• Figure 1: Overview of our proposed sentence-level Process Reward Model (PRM) for verifying radiology reports. The PRM takes clinical context and previously generated/verified sentences as input to predict the correctness of the current sentence, enabling fine-grained hallucination detection.
• Figure 2: Impact of rejecting lowest-scoring reports (x-axis) on clinical factuality metrics (y-axis) of remaining reports. PRM-avg and PRM-prod show steepest improvements, indicating effective quality filtering. Entropy also shows good filtering quality for F1-RadGraph scores, sometimes outperforming PRM methods at high rejection thresholds.
• Figure 3: Performance of Weighted BoN strategies vs. N (temp=1.0). W-Avg and W-Min show strong improvement. Scoring strategies W-Avg, W-Min, W-Prod, and W-Log refer to weighted BoN using base scores from AvgProb, MinProb, ProdProb, and LogProb, respectively.
• Figure 4: Comparison of RadNLI Yuan2021 and GPT-4o for entailment labeling, used for generating weak labels. (a) Performance metrics evaluated on the RadNLI test dataset. (b) Prompt used for GPT-4o evaluation on the RadNLI test dataset.
• Figure 5: Example PRM input structure for training, using report mimic-54422699 from the MIMIC-CXR dataset. The input includes clinical context (Indication, Technique, Comparison), MAIRA-2 generated sentences, and ground-truth labels (interleaved during training only). For inference, labels are omitted and predicted by the PRM.