Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Data

Mengmeng Zhang, Xiaoping Wu, Hao Luo, Fan Wang, Yisheng Lv

TL;DR

MedGround presents a mask-guided synthesis and verification pipeline that converts segmentation annotations into image–text–box triplets for medical referring grounding. By deriving precise bounding boxes from expert masks and enforcing multi-stage verification, it produces high-quality, clinically grounded queries that couple morphology and location with visual evidence. The resulting MedGround-35K dataset enables substantial improvements in grounding accuracy, semantic alignment, and zero-shot generalization across diverse medical imaging modalities and models. This scalable approach anchors medical language to verifiable visual targets, enhancing interpretability and robustness of medical VLMs, with dataset and code to be released after acceptance.

Abstract

Vision-Language Models (VLMs) can generate convincing clinical narratives, yet frequently struggle to visually ground their statements. We posit this limitation arises from the scarcity of high-quality, large-scale clinical referring-localization pairs. To address this, we introduce MedGround, an automated pipeline that transforms segmentation resources into high-quality medical referring grounding data. Leveraging expert masks as spatial anchors, MedGround precisely derives localization targets, extracts shape and spatial cues, and guides VLMs to synthesize natural, clinically grounded queries that reflect morphology and location. To ensure data rigor, a multi-stage verification system integrates strict formatting checks, geometry- and medical-prior rules, and image-based visual judging to filter out ambiguous or visually unsupported samples. Finally, we present MedGround-35K, a novel multimodal medical dataset. Extensive experiments demonstrate that VLMs trained with MedGround-35K consistently achieve improved referring grounding performance, enhance multi-object semantic disambiguation, and exhibit strong generalization to unseen grounding settings. This work highlights MedGround as a scalable, data-driven approach to anchor medical language to verifiable visual evidence. Dataset and code will be released publicly upon acceptance.

MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Data

TL;DR

MedGround presents a mask-guided synthesis and verification pipeline that converts segmentation annotations into image–text–box triplets for medical referring grounding. By deriving precise bounding boxes from expert masks and enforcing multi-stage verification, it produces high-quality, clinically grounded queries that couple morphology and location with visual evidence. The resulting MedGround-35K dataset enables substantial improvements in grounding accuracy, semantic alignment, and zero-shot generalization across diverse medical imaging modalities and models. This scalable approach anchors medical language to verifiable visual targets, enhancing interpretability and robustness of medical VLMs, with dataset and code to be released after acceptance.

Abstract

Vision-Language Models (VLMs) can generate convincing clinical narratives, yet frequently struggle to visually ground their statements. We posit this limitation arises from the scarcity of high-quality, large-scale clinical referring-localization pairs. To address this, we introduce MedGround, an automated pipeline that transforms segmentation resources into high-quality medical referring grounding data. Leveraging expert masks as spatial anchors, MedGround precisely derives localization targets, extracts shape and spatial cues, and guides VLMs to synthesize natural, clinically grounded queries that reflect morphology and location. To ensure data rigor, a multi-stage verification system integrates strict formatting checks, geometry- and medical-prior rules, and image-based visual judging to filter out ambiguous or visually unsupported samples. Finally, we present MedGround-35K, a novel multimodal medical dataset. Extensive experiments demonstrate that VLMs trained with MedGround-35K consistently achieve improved referring grounding performance, enhance multi-object semantic disambiguation, and exhibit strong generalization to unseen grounding settings. This work highlights MedGround as a scalable, data-driven approach to anchor medical language to verifiable visual evidence. Dataset and code will be released publicly upon acceptance.
Paper Structure (32 sections, 10 figures, 12 tables)

This paper contains 32 sections, 10 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Medical Vision–Language Models
  4. Referring Expression Grounding
  5. VLMs-based Synthesis and Verification
  6. MedGround
  7. Dataset Definition
  8. From Masks to Ground-Truth Box Lists
  9. Mask-Guided Query Synthesis
  10. Multi-Stage Data Verification and Cleaning
  11. MedGround-35K
  12. Human Audit
  13. Experiments
  14. Experiment Settings
  15. Results and Analysis
  16. ...and 17 more sections

Figures (10)

  • Figure 1: Motivation of MedGround. (a) Models trained on image-text pairs fail to "speak with substance" due to lack of grounding. (b) Segmentation-only training fails to achieve semantic understanding. (c) MedGround (Image-text-box triplets) activates the full potential of medical VLMs by bridging semantics and localization.
  • Figure 2: MedGround pipeline. (A) Convert segmentation masks into normalized ground-truth bounding box lists. (B) Use dataset-aware, mask-guided prompts to synthesize medically meaningful referring queries and select target box(es) as answers. (C) Perform multi-stage verification and cleaning (format/schema, geometry–location rules, and VLM-based grounding). (D) Conduct manual review for final quality control. (E) Cases falling verification.
  • Figure 3: Diversity and Linguistic Complexity of the MedGround dataset. Up: The word cloud illustrates the distribution of medical terminology, anatomical landmarks, and clinical descriptors within the grounding annotations. Down: Comparative analysis of annotation richness across five distinct modalities based on three key metrics: (a) Clinical Entity Density, (b) Morphology Term Coverage, and (c) Spatial Relation Complexity.
  • Figure 4: Label-based Dataset vs. MedGround-35K. (A) Label-based Dataset: conventional datasets typically group all ground-truth boxes under a single generic category label (e.g., "lesion") for the entire image. (B) MedGround-35K: provides distinct, fine-grained descriptive expressions for each localized box, capturing specific clinical nuances for individual regions.
  • Figure 5: Examples of the Semantic Sensitivity Testing Dataset
  • ...and 5 more figures