Med-Scout: Curing MLLMs' Geometric Blindness in Medical Perception via Geometry-Aware RL Post-Training

TL;DR

Med-Scout addresses the geometric grounding gap in medical Multimodal LLMs by employing geometry-aware reinforcement learning post-training. It formulates three verifiable geometric proxy tasks—Hierarchical Scale Localization, Topological Jigsaw Reconstruction, and Anomaly Consistency Detection—and optimizes them with a Dense Geometric Reward within a GRPO framework, using unlabeled medical images to induce objective supervision. The approach yields over 40% gains on the Med-Scout-Bench and generalizes across radiological VQA and broader medical perception benchmarks, aided by energy-based analyses and targeted attention improvements. This framework enables reliable geometric reasoning in medical perception, reducing hallucinations while leveraging unlabeled data and scalable training.

Abstract

Despite recent Multimodal Large Language Models (MLLMs)' linguistic prowess in medical diagnosis, we find even state-of-the-art MLLMs suffer from a critical perceptual deficit: geometric blindness. This failure to ground outputs in objective geometric constraints leads to plausible yet factually incorrect hallucinations, rooted in training paradigms that prioritize linguistic fluency over geometric fidelity. This paper introduces Med-Scout, a novel framework that "cures" this blindness via Reinforcement Learning (RL) that leverages the intrinsic geometric logic latent within unlabeled medical images. Instead of relying on costly expert annotations, Med-Scout derives verifiable supervision signals through three strategic proxy tasks: Hierarchical Scale Localization, Topological Jigsaw Reconstruction, and Anomaly Consistency Detection. To rigorously quantify this deficit, we present Med-Scout-Bench, a new benchmark specifically designed to evaluate geometric perception. Extensive evaluations show that Med-Scout significantly mitigates geometric blindness, outperforming leading proprietary and open-source MLLMs by over 40% on our benchmark. Furthermore, this enhanced geometric perception generalizes to broader medical understanding, achieving superior results on radiological and comprehensive medical VQA tasks.

Figures (13)

• Figure 1: Pilot Study: (Left) Scale Blindness: The model correctly describes findings in a local crop but fails in the full global view. (Middle) Topology Blindness: When the image is rotated, the model fails to update location descriptions. (Right) Anomaly Blindness: The model completely overlooks obvious artificial modifications.
• Figure 2: Overview of the Med-Scout Framework. We transform unlabeled medical images into three proxy tasks to cure geometric blindness actively. The framework is optimized using GRPO with a Dense Geometric Reward mechanism that provides stable feedback.
• Figure 3: Performance comparison on Med-Scout-Bench.
• Figure 4: Performance comparison on six public benchmarks. Purple colors correspond to higher accuracy. Direct Mode (+M) and Reasoning Mode (+M (R)) show close performance.
• Figure 5: Data Scaling and Generalization Analysis. Left: Continuous performance improvement on Med-Scout-Bench with increasing training data. Right: Strong positive correlation between Med-Scout-Bench scores and average accuracy on external benchmarks.