MedGRPO: Multi-Task Reinforcement Learning for Heterogeneous Medical Video Understanding

TL;DR

The paper tackles the difficulty of medical video understanding by introducing MedVidBench, a large-scale benchmark across eight medical sources and eight tasks, and MedGRPO, a balanced multi-dataset reinforcement learning framework. MedGRPO combines cross-dataset reward normalization with a medical LLM judge to stabilize training and improve captioning and grounding tasks across heterogeneous data. Empirical results show supervised fine-tuning on MedVidBench outperforms existing closed-source models, and MedGRPO further enhances performance while preventing optimization collapse. Together, these contributions establish a scalable benchmark and a robust training paradigm for vision-language models in medical domains, enabling more reliable clinical video analysis.

Abstract

Large vision-language models struggle with medical video understanding, where spatial precision, temporal reasoning, and clinical semantics are critical. To address this, we first introduce \textbf{MedVidBench}, a large-scale benchmark of 531,850 video-instruction pairs across 8 medical sources spanning video, segment, and frame-level tasks, curated through a rigorous quality assurance pipeline with expert-guided prompting and dual-model validation. While supervised fine-tuning on MedVidBench yields noticeable gains, standard Reinforcement Learning (RL) fails due to imbalanced reward scales across datasets, which destabilizes optimization and leads to training collapse. To overcome this, we introduce \textbf{MedGRPO}, a novel RL framework for balanced multi-dataset training with two key innovations: (1) \emph{cross-dataset reward normalization} that maps each dataset's median performance to a common reward value, ensuring fair optimization regardless of difficulty, and (2) a \emph{medical LLM judge} that evaluates caption quality on five clinical dimensions through comparative similarity scoring. Supervised fine-tuning Qwen2.5-VL-7B on MedVidBench substantially outperforms GPT-4.1 and Gemini-2.5-Flash across all tasks, demonstrating MedVidBench's efficacy, while our MedGRPO framework further improves upon the SFT baseline across grounding and captioning tasks. Our work establishes a foundational benchmark and robust training methodology for advancing vision-language models in medical domains. Our project website is available at https://yuhaosu.github.io/MedGRPO/.

Figures (11)

• Figure 1: Overview of MedVidBench. (a) High quality data curation pipeline for MedVidBench. We leaverage expert knowledge into prompt construction and generate high quality text using 2 VLMs (Gemini-2.5-Flash and GPT-4.1). (b) MedVidBench comprises of 8 different datasets, with 532k samples in total, spanning 4 different domains. (c) Examples of diverse tasks across different domains.
• Figure 2: Overview of MedGRPO. (a) MedGRPO framework with cross-dataset reward normalization and medical LLM judge evaluation. (b) Training entropy comparison between models trained with and without reward normalization.
• Figure 3: Qualitative comparison of region captioning generation.
• Figure 4: Scaling law analysis. Performance on Dense Video Captioning (DVC F1 score) and Video Summarization (VS LLM judge score) improves consistently as training samples from the Large-Scale version increase from 0 to 461K.
• Figure 5: Interface for human validation study. Users were provided detailed instruction to rank caption after watching a short video. An instruction example for a good and bad caption was provided.