Can multiple-choice questions really be useful in detecting the abilities of LLMs?

TL;DR

The paper investigates whether multiple-choice questions (MCQs) reliably measure large language models' (LLMs) capabilities, especially in knowledge-intensive tasks where long-form generation (LFGQ) is common. It analyzes nine LLMs across four QA benchmarks in Chinese and English, uncovering a pronounced order sensitivity bias favoring the first option in bilingual MCQs and a measurable misalignment between MCQ and LFGQ representations across outputs, token logits, and embeddings. The authors introduce a unified confidence framework and an expected calibration error (ECE) metric to compare formats, finding MCQs are less calibrated than LFGQs and that higher response consistency does not imply higher accuracy. They conclude that LFGQs better reflect real-world use and provide recommendations to improve evaluation practices, including promoting LFGQs in high-stakes settings and employing cross-format analyses to mitigate format-specific biases. The work emphasizes embedding-space and calibration-based diagnostics as essential tools for robust LLM evaluation and offers open-source code and models for reproducibility.

Abstract

Multiple-choice questions (MCQs) are widely used in the evaluation of large language models (LLMs) due to their simplicity and efficiency. However, there are concerns about whether MCQs can truly measure LLM's capabilities, particularly in knowledge-intensive scenarios where long-form generation (LFG) answers are required. The misalignment between the task and the evaluation method demands a thoughtful analysis of MCQ's efficacy, which we undertake in this paper by evaluating nine LLMs on four question-answering (QA) datasets in two languages: Chinese and English. We identify a significant issue: LLMs exhibit an order sensitivity in bilingual MCQs, favoring answers located at specific positions, i.e., the first position. We further quantify the gap between MCQs and long-form generation questions (LFGQs) by comparing their direct outputs, token logits, and embeddings. Our results reveal a relatively low correlation between answers from MCQs and LFGQs for identical questions. Additionally, we propose two methods to quantify the consistency and confidence of LLMs' output, which can be generalized to other QA evaluation benchmarks. Notably, our analysis challenges the idea that the higher the consistency, the greater the accuracy. We also find MCQs to be less reliable than LFGQs in terms of expected calibration error. Finally, the misalignment between MCQs and LFGQs is not only reflected in the evaluation performance but also in the embedding space. Our code and models can be accessed at https://github.com/Meetyou-AI-Lab/Can-MC-Evaluate-LLMs.

Figures (13)

• Figure 1: Example of order sensitivity experiments, in which the correct answer is D. In the ARC dataset, when predicting a wrong option (A, B, or C), the LLM prefers the option located in the first position.
• Figure 2: Comparison between MCQs and LFGQs on the CARE-MI dataset. Top: accuracy and pearson correlation. Bottom: MCQ vs LFGQs matrix.
• Figure 3: GPT-3.5-turbo's consistency on MCQs and LFGQs with different temperatures.
• Figure 4: Reliability diagrams for MCQs, LFGQs, and TFQs on the CARE-MI dataset.
• Figure 5: t-SNE visualization for each layer in Llama-2-7b-chat-hf. MCQs in red, LFGQs in green.