Beyond Multiple Choice: Verifiable OpenQA for Robust Vision-Language RFT

TL;DR

This work identifies critical fragility in MCQA as an evaluation and training format for vision-language models, where distractors and option layouts introduce signals that inflate performance and encourage shortcut strategies. It introduces ReVeL, a three-phase rewrite-and-verify framework that converts MCQA into OpenQA with a hybrid verifier combining rule-based checks and selective LLM judgments, achieving high rule-verifiability on multiple benchmarks. When used for training, ReVeL-enabled OpenQA data improves open-ended accuracy while preserving MCQA performance, demonstrating better data efficiency and more robust, transferable reasoning. The approach also reveals substantial score inflation in MCQA benchmarks during evaluation and highlights the potential of verifiable OpenQA to align training signals with real-world open-ended tasks. The work provides a practical, scalable path toward robust vision-language evaluation and reinforcement learning, with public release of code and data.

Abstract

Multiple-choice question answering (MCQA) has been a popular format for evaluating and reinforcement fine-tuning (RFT) of modern multimodal language models. Its constrained output format allows for simplified, deterministic automatic verification. However, we find that the options may leak exploitable signals, which makes the accuracy metrics unreliable for indicating real capabilities and encourages explicit or implicit answer guessing behaviors during RFT. We propose ReVeL (Rewrite and Verify by LLM), a framework that rewrites multiple-choice questions into open-form questions while keeping answers verifiable whenever possible. The framework categorizes questions according to different answer types, apply different rewriting and verification schemes, respectively. When applied for RFT, we converted 20k MCQA examples and use GRPO to finetune Qwen2.5-VL models. Models trained on ReVeL-OpenQA match MCQA accuracy on multiple-choice benchmarks and improve OpenQA accuracy by about six percentage points, indicating better data efficiency and more robust reward signals than MCQA-based training. When used for evaluation, ReVeL also reveals up to 20 percentage points of score inflation in MCQA benchmarks (relative to OpenQA), improves judging accuracy, and reduces both cost and latency. We will release code and data publicly.

Figures (18)

• Figure 1: Illustration of MCQA fragility. The example (left) shows an unfaithful reasoning chain that eliminates distractors incorrectly yet provide a correct final answer, yielding a positive reward signal that, when used in reinforcement learning, further amplifies shortcut behavior (top right). This shortcut behavior leads to widening gap between MCQA and OpenQA. The diagram motivate us to propose ReVeL, which aligns evaluation and training with reliable OpenQA.
• Figure 2: Performance comparison on original open-ended datasets (SimpleQA, Visual SimpleQA) and their multiple-choice versions (*-Choice, with 6 options). The Random Guess score is a theoretical upper bound that combines the model's actual open-ended accuracy with the probability of correctly guessing on the rest of the questions from six options.
• Figure 3: Reasoning and answer can mismatch after replacing the ground-truth option with NOTA.
• Figure 4: On the impact of options on multiple-choice benchmarks: when options are removed, accuracy is uniformly lower, especially on VQA benchmarks like MMMU.
• Figure 5: Illustration of the rewrite-and-verify framework