Is Your Large Language Model Knowledgeable or a Choices-Only Cheater?

TL;DR

The paper investigates whether MCQA leaderboard rankings primarily reflect genuine knowledge or reliance on choices-only shortcuts. It introduces a graph-mining based automatic contrast-set construction to generate a $${820}$$-question contrast set from UnifiedQA, enabling a robust test of choices-only effects. Across $12$ LLMs and varying $5$- and $10$-shot prompts, it finds high consistency between full-question and choices-only rankings, quantified by Kendall's $\tau$ values of $0.88$ and $0.91$, indicating that leaderboard performance is not simply due to exploiting choices-only shortcuts. The work presents a scalable methodology for probing LLM decision-making and supports the validity of MCQA as a measure of knowledge, while highlighting the need to understand the emergence of high choices-only accuracy.

Abstract

Recent work shows that large language models (LLMs) can answer multiple-choice questions using only the choices, but does this mean that MCQA leaderboard rankings of LLMs are largely influenced by abilities in choices-only settings? To answer this, we use a contrast set that probes if LLMs over-rely on choices-only shortcuts in MCQA. While previous works build contrast sets via expensive human annotations or model-generated data which can be biased, we employ graph mining to extract contrast sets from existing MCQA datasets. We use our method on UnifiedQA, a group of six commonsense reasoning datasets with high choices-only accuracy, to build an 820-question contrast set. After validating our contrast set, we test 12 LLMs, finding that these models do not exhibit reliance on choice-only shortcuts when given both the question and choices. Thus, despite the susceptibility~of MCQA to high choices-only accuracy, we argue that LLMs are not obtaining high ranks on MCQA leaderboards just due to their ability to exploit choices-only shortcuts.

Figures (6)

• Figure 2: Distractor plausibility ratings across methods.
• Figure 3: Accuracy of twelve LLMs on the UnifiedQA evaluation set (left) versus its contrast set (right), sorted by full prompt accuracy. We show 5-shot (top) and 10-shot (bottom) prompts, with 3-shot prompts in Appendix \ref{['appendix_3_shot']}.
• Figure 4: Instructions shown to annotators.
• Figure 5: Interface used by annotators.
• Figure 6: 3-shot benchmarking of 12 LLMs on the UnifiedQA evaluation set and the contrast set, sorted by full-prompt accuracy. The same trends found for 5-shot and 10-shot prompting hold for 3-shot prompting.