MicroVQA++: High-Quality Microscopy Reasoning Dataset with Weakly Supervised Graphs for Multimodal Large Language Model

TL;DR

MicroVQA++ tackles the data bottleneck in microscopy reasoning for MLLMs by building a large, high-quality multimodal corpus from BIOMEDICA through a three-stage pipeline. It introduces HiCQA-Graph, a heterogeneous image–caption–QA graph that enforces cross-modal consistency via NLI entailment and CLIP-based alignment, followed by human screening to ensure reliability. After supervised fine-tuning of a 4B-parameter MLLM on MicroVQA++ data, the model matches the performance of GPT-5 on the MicroVQA benchmark and sets open-source SOTA in microscopy reasoning, while the test set remains harder and less overlapping with the training distribution. This work demonstrates that careful data construction and graph-based filtering can unlock substantial gains for domain-specific reasoning in microscopy, enabling smaller models to excel in specialized VQA tasks.

Abstract

Multimodal Large Language Models are increasingly applied to biomedical imaging, yet scientific reasoning for microscopy remains limited by the scarcity of large-scale, high-quality training data. We introduce MicroVQA++, a three-stage, large-scale and high-quality microscopy VQA corpus derived from the BIOMEDICA archive. Stage one bootstraps supervision from expert-validated figure-caption pairs sourced from peer-reviewed articles. Stage two applies HiCQA-Graph, a novel heterogeneous graph over images, captions, and QAs that fuses NLI-based textual entailment, CLIP-based vision-language alignment, and agent signals to identify and filter inconsistent samples. Stage three uses a MultiModal Large Language Model (MLLM) agent to generate multiple-choice questions (MCQ) followed by human screening. The resulting release comprises a large training split and a human-checked test split whose Bloom's level hard-sample distribution exceeds the MicroVQA benchmark. Our work delivers (i) a quality-controlled dataset that couples expert literature with graph-based filtering and human refinement; (ii) HiCQA-Graph, the first graph that jointly models (image, caption, QA) for cross-modal consistency filtering; (iii) evidence that careful data construction enables 4B-scale MLLMs to reach competitive microscopy reasoning performance (e.g., GPT-5) and achieve state-of-the-art performance among open-source MLLMs. Code and dataset will be released after the review process concludes.

Figures (4)

• Figure 1: Bloom's levels across microscopy multimodal datasets. MicroVQA++ exhibits a substantially higher proportion in harder level and absolute count of higher-difficulty questions than MicroVQA, reflecting a stricter and more demanding evaluation setting.
• Figure 2: MicroVQA++ is built in three stages. We first sample figure–caption pairs from the Microscopy category of BIOMEDICA. An MLLM agent then extracts answer spans from the captions to construct initial QA pairs, leveraging peer-reviewed articles to ensure expert-validated supervision. Next, we pass the data through HiCQA-Graph, which evaluates cross-modal consistency to judge generation quality. Finally, conditioned on the validated items (human check pipeline available in appendix), an MLLM agent produces CoT rationales and MCQ variants for each question. red indicates grounding informations and green indicates important entities.
• Figure 3: a) and b) shows CLIP and NLI only filtering method. c) indicates HiCQA-Graph structure. Cross-modal consistency token is added to Image and QA nodes. Two heads are used to predict soft weak supervised labels.
• Figure 4: Two-dimensional t-SNE of CLIP embeddings for images and questions. t-SNE uses perplexity of 30 and 1,000 iterations.