Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differentiating Choices via Commonality for Multiple-Choice Question Answering

Wenqing Deng, Zhe Wang, Kewen Wang, Shirui Pan, Xiaowang Zhang, Zhiyong Feng

TL;DR

This work tackles MCQA when all answer choices are relevant and semantically similar by introducing Differentiating Choices via Commonality for MCQA (DCQA). DCQA identifies the common information shared by all choices and subtracts it to form choice-specific question representations, then uses a decoder to generate discriminative context and refines each choice accordingly. Empirical results across five benchmarks show DCQA consistently improves over encoder-decoder baselines, with ablations confirming the necessity of each module and cross-attention mechanism, and a case study illustrating improved interpretability through differentiating clues. The method does not rely on external knowledge bases and demonstrates robust performance and explainability across diverse MCQA tasks, offering practical benefits for interpretable reasoning in QA systems.

Abstract

Multiple-choice question answering (MCQA) becomes particularly challenging when all choices are relevant to the question and are semantically similar. Yet this setting of MCQA can potentially provide valuable clues for choosing the right answer. Existing models often rank each choice separately, overlooking the context provided by other choices. Specifically, they fail to leverage the semantic commonalities and nuances among the choices for reasoning. In this paper, we propose a novel MCQA model by differentiating choices through identifying and eliminating their commonality, called DCQA. Our model captures token-level attention of each choice to the question, and separates tokens of the question attended to by all the choices (i.e., commonalities) from those by individual choices (i.e., nuances). Using the nuances as refined contexts for the choices, our model can effectively differentiate choices with subtle differences and provide justifications for choosing the correct answer. We conduct comprehensive experiments across five commonly used MCQA benchmarks, demonstrating that DCQA consistently outperforms baseline models. Furthermore, our case study illustrates the effectiveness of the approach in directing the attention of the model to more differentiating features.

Differentiating Choices via Commonality for Multiple-Choice Question Answering

TL;DR

This work tackles MCQA when all answer choices are relevant and semantically similar by introducing Differentiating Choices via Commonality for MCQA (DCQA). DCQA identifies the common information shared by all choices and subtracts it to form choice-specific question representations, then uses a decoder to generate discriminative context and refines each choice accordingly. Empirical results across five benchmarks show DCQA consistently improves over encoder-decoder baselines, with ablations confirming the necessity of each module and cross-attention mechanism, and a case study illustrating improved interpretability through differentiating clues. The method does not rely on external knowledge bases and demonstrates robust performance and explainability across diverse MCQA tasks, offering practical benefits for interpretable reasoning in QA systems.

Abstract

Multiple-choice question answering (MCQA) becomes particularly challenging when all choices are relevant to the question and are semantically similar. Yet this setting of MCQA can potentially provide valuable clues for choosing the right answer. Existing models often rank each choice separately, overlooking the context provided by other choices. Specifically, they fail to leverage the semantic commonalities and nuances among the choices for reasoning. In this paper, we propose a novel MCQA model by differentiating choices through identifying and eliminating their commonality, called DCQA. Our model captures token-level attention of each choice to the question, and separates tokens of the question attended to by all the choices (i.e., commonalities) from those by individual choices (i.e., nuances). Using the nuances as refined contexts for the choices, our model can effectively differentiate choices with subtle differences and provide justifications for choosing the correct answer. We conduct comprehensive experiments across five commonly used MCQA benchmarks, demonstrating that DCQA consistently outperforms baseline models. Furthermore, our case study illustrates the effectiveness of the approach in directing the attention of the model to more differentiating features.
Paper Structure (22 sections, 7 equations, 4 figures, 7 tables)

This paper contains 22 sections, 7 equations, 4 figures, 7 tables.

Table of Contents

  1. Related Work
  2. Knowledge-Enhanced Models on MCQA Tasks
  3. Contextual Information Models on MCQA Task
  4. Method
  5. Context Representation
  6. Commonality Extraction
  7. Context Refinement
  8. Choice Enhancement
  9. Experiments
  10. Datasets
  11. Experimental Setup
  12. Baselines
  13. Main Results and Analysis
  14. Ablation Study
  15. Case Study
  16. ...and 7 more sections

Figures (4)

  • Figure 1: An example of MCQA with its reasoning process to derive the correct choice B. The red tokens in the question are common to all choices and the blue tokens are differentiating ones for choosing the answer.
  • Figure 2: Overall architecture of our proposed DCQA model. We input the Q context (question) as the representation of the question and the QA context (question + choice) as the representation of each choice.
  • Figure 3: The comparison of weights visualization between the GenMC model and our model on three examples from the Dev dataset of CSQA. The correct choice is marked with *. The darker the color is, the higher the weight is.
  • Figure 4: The heatmaps of two examples from the Dev dataset of CSQA.