MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation

TL;DR

MedRAGChecker tackles the risk of unsafe or unsupported biomedical outputs in retrieval-augmented generation by decomposing long-form answers into atomic claims and verifying each claim with a dual signal: textual NLI from teacher-distilled checkers and a KG-based consistency signal from DRKG, fused into a calibrated per-claim score $P^\star$ that drives diagnostics. The approach is distilled into compact student models for claim extraction and verification to enable scalable evaluation, validated against human judgments. Experiments on PubMedQA, MedQuAD, LiveQA, and MedRedQA reveal that MedRAGChecker reliably flags unsupported and contradicted claims and uncovers distinct risk profiles across generators, especially for safety-critical relations. The work provides a practical, scalable framework for claim-level grounding in biomedical RAG, enhancing safety and interpretability in real-world medical QA systems.

Abstract

Biomedical retrieval-augmented generation (RAG) can ground LLM answers in medical literature, yet long-form outputs often contain isolated unsupported or contradictory claims with safety implications. We introduce MedRAGChecker, a claim-level verification and diagnostic framework for biomedical RAG. Given a question, retrieved evidence, and a generated answer, MedRAGChecker decomposes the answer into atomic claims and estimates claim support by combining evidence-grounded natural language inference (NLI) with biomedical knowledge-graph (KG) consistency signals. Aggregating claim decisions yields answer-level diagnostics that help disentangle retrieval and generation failures, including faithfulness, under-evidence, contradiction, and safety-critical error rates. To enable scalable evaluation, we distill the pipeline into compact biomedical models and use an ensemble verifier with class-specific reliability weighting. Experiments on four biomedical QA benchmarks show that MedRAGChecker reliably flags unsupported and contradicted claims and reveals distinct risk profiles across generators, particularly on safety-critical biomedical relations.

Figures (4)

• Figure 1: Example of claim-level verification in biomedical RAG. We decompose both the model response and the reference answer into atomic claims. Each model claim is labeled as Entail (supported) or Contradict given retrieved evidence, and reference claims not covered by the model are marked as Missing.
• Figure 2: Overview of MedRAGChecker. For each atomic claim, textual NLI checking (purple) and KG-based verification (orange) provide complementary signals---semantic entailment from retrieved context versus structured biomedical constraints---which are fused and distilled into a student verifier (grey).
• Figure 3: Example where KG support corrects an over-confident textual entailment decision.
• Figure 4: Sensitivity of KG--NLI fusion to the mixing weight $\beta$. Top: fused entailment rate (fused_E), i.e., the fraction of claims classified as supported after fusion and thresholding. Bottom: decision flip rate (flip_rate) relative to a near-NLI reference setting ($\beta{=}0.9$), measuring how often KG changes the supported/not-supported decision. We compare SFT and GRPO checkers, with and without KG score calibration (minmax vs. none).