CURE: Curriculum-guided Multi-task Training for Reliable Anatomy Grounded Report Generation

TL;DR

CURE tackles unreliable grounding and factuality in medical vision-language models by introducing an error-aware curriculum that dynamically reweights multi-task training across anatomy-centered tasks, all using public datasets. By reformulating tasks into a unified fine-grained instructional format and applying inter- and intra-dataset sampling guided by current errors, CURE substantially improves visual grounding (IoU) and factual consistency (CXRFEScore) while reducing hallucinations by 18.6%. The framework transfers grounding capabilities to base models that lack grounding, achieving state-of-the-art performance on several benchmarks including Chest ImaGenome and zero-shot VinDr-CXR, with notable gains even when data is limited. The approach is data-efficient, data-agnostic to new domains, and comes with accessible code and weights, offering a practical path to more trustworthy radiology report generation at scale.

Abstract

Medical vision-language models can automate the generation of radiology reports but struggle with accurate visual grounding and factual consistency. Existing models often misalign textual findings with visual evidence, leading to unreliable or weakly grounded predictions. We present CURE, an error-aware curriculum learning framework that improves grounding and report quality without any additional data. CURE fine-tunes a multimodal instructional model on phrase grounding, grounded report generation, and anatomy-grounded report generation using public datasets. The method dynamically adjusts sampling based on model performance, emphasizing harder samples to improve spatial and textual alignment. CURE improves grounding accuracy by +0.37 IoU, boosts report quality by +0.188 CXRFEScore, and reduces hallucinations by 18.6%. CURE is a data-efficient framework that enhances both grounding accuracy and report reliability. Code is available at https://github.com/PabloMessina/CURE and model weights at https://huggingface.co/pamessina/medgemma-4b-it-cure

Figures (7)

• Figure 1: False Positive Detection of Pathologies. Given the same chest X-ray input both models approximate the location of the left clavicle. However, the baseline model (MAIRA-2) hallucinates a fracture (there is no fracture in the image), whereas our proposed model (CURE) generates a clinically correct and visually grounded description.
• Figure 2: Overview of CURE, our Curriculum-guided Multi-task Training Framework. During training, the model is periodically evaluated every N steps on validation subsets from each task. Performance metrics (IoU, CXRFEScore) are calculated to identify task-level and category-level errors, which are then used to update the sampling weights in the training sampler. The cycle then resumes, allowing the model to focus more heavily on the data it finds most challenging. Evaluation of the RG task uses the official MIMIC-CXR test set, while VinDr-CXR is assessed in a zero-shot setting.
• Figure 3: Qualitative Results. Qualitative phrase grounding (PG) results for the prompt "Chronic inflammatory changes predominantly in both lung apices," shown on an example from the PadChest-GR dataset padchest-gr. Ground-truth regions are shown in green across all images for reference. Model predictions from CURE and MAIRA-2 are shown in red. The MedGemma-4B-IT model does not output visual grounding and therefore displays no predicted regions.
• Figure 4: Visualization of Inter-Dataset Weight Dynamics. This plot illustrates the curriculum's adaptation from an experiment with frequent updates (every 500 steps). It shows how sampling probabilities for each data source evolve over time in response to the model's performance.
• Figure 5: Visualization of Intra-Dataset Weight Dynamics for MS-CXR. This plot shows the category-level weight evolution for the 8 phrase classes in MS-CXR, taken from the same experiment with updates every 500 steps. The weights are periodically adjusted to prioritize classes with higher error rates.