Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Difficulty-Controllable Multiple-Choice Question Generation Using Large Language Models and Direct Preference Optimization

Yuto Tomikawa, Masaki Uto

TL;DR

The paper tackles the problem of generating difficulty-controllable multiple-choice questions (MCQs) for reading comprehension. It introduces a method that uses an open-source LLM (Llama 3.1) fine-tuned with supervised learning, followed by Direct Preference Optimization (DPO) to explicitly maximize the accuracy of difficulty control, with difficulty levels anchored by the Rasch model from Item Response Theory (IRT). A dataset with (context, question, correct option, distractors, difficulty) is constructed by simulating QA systems of varying ability to estimate question difficulty, enabling training for MCQ generation across a spectrum of levels. Experiments on the RACE dataset show that DPO improves the alignment between specified and generated difficulty, while preserving fluency, relevance, and answerability; few-shot attempts for difficulty control prove ineffective, underscoring the value of the proposed two-stage SFT+DPO approach. The work advances adaptive learning support by enabling MCQ generation tailored to learner ability and supports more accurate ability estimation via higher Fisher information near target abilities, with implications for scalable, ability-aligned assessment design.

Abstract

Difficulty-controllable question generation for reading comprehension has gained significant attention in the field of education as a fundamental tool for adaptive learning support. Although several neural question generation methods have recently succeeded in controlling difficulty, conventional approaches still face two major limitations. First, they cannot directly generate multiple-choice questions, which are the most widely used question type in educational contexts. Second, they are not explicitly trained to optimize the accuracy of difficulty control, leaving room for further improvement in difficulty controllability. To address these limitations, this study proposes a novel difficulty-controllable multiple-choice question generation method for reading comprehension which leverages a large language model trained using a direct preference optimization technique to improve the accuracy of difficulty control.

Difficulty-Controllable Multiple-Choice Question Generation Using Large Language Models and Direct Preference Optimization

TL;DR

The paper tackles the problem of generating difficulty-controllable multiple-choice questions (MCQs) for reading comprehension. It introduces a method that uses an open-source LLM (Llama 3.1) fine-tuned with supervised learning, followed by Direct Preference Optimization (DPO) to explicitly maximize the accuracy of difficulty control, with difficulty levels anchored by the Rasch model from Item Response Theory (IRT). A dataset with (context, question, correct option, distractors, difficulty) is constructed by simulating QA systems of varying ability to estimate question difficulty, enabling training for MCQ generation across a spectrum of levels. Experiments on the RACE dataset show that DPO improves the alignment between specified and generated difficulty, while preserving fluency, relevance, and answerability; few-shot attempts for difficulty control prove ineffective, underscoring the value of the proposed two-stage SFT+DPO approach. The work advances adaptive learning support by enabling MCQ generation tailored to learner ability and supports more accurate ability estimation via higher Fisher information near target abilities, with implications for scalable, ability-aligned assessment design.

Abstract

Difficulty-controllable question generation for reading comprehension has gained significant attention in the field of education as a fundamental tool for adaptive learning support. Although several neural question generation methods have recently succeeded in controlling difficulty, conventional approaches still face two major limitations. First, they cannot directly generate multiple-choice questions, which are the most widely used question type in educational contexts. Second, they are not explicitly trained to optimize the accuracy of difficulty control, leaving room for further improvement in difficulty controllability. To address these limitations, this study proposes a novel difficulty-controllable multiple-choice question generation method for reading comprehension which leverages a large language model trained using a direct preference optimization technique to improve the accuracy of difficulty control.
Paper Structure (18 sections, 5 equations, 6 figures, 7 tables)

This paper contains 18 sections, 5 equations, 6 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Neural QG for Reading Comprehension
  4. Difficulty-Controllable Neural QG for Reading Comprehension
  5. Difficulty-Controllable QG Using Item Response Theory
  6. Proposed Method
  7. Constructing a Multiple-Choice Question Dataset with Difficulty Levels
  8. Initial Fine-tuning
  9. Direct Preference Optimization for Improving Difficulty Control Accuracy
  10. Experiments
  11. Experimental Procedures
  12. Evaluation of Accuracy of Difficulty Control
  13. Evaluation based on Fisher Information
  14. Evaluation of the Fluency, Content Relevance, and Answerability
  15. Evaluation of Difficulty Controllability in Few-Shot Learning
  16. ...and 3 more sections

Figures (6)

  • Figure 1: Conceptual diagram of answer-extractive QG (task in previous studies tomikawa2024adaptive) and multiple-choice QG (task in the present study).
  • Figure 2: Outline of RLHF for optimizing the accuracy of difficulty control.
  • Figure 3: Relationship between specified difficulty and correct response rates.
  • Figure 4: Relationship between specified and estimated IRT-based difficulty.
  • Figure 5: Fisher information curves for questions generated at the specified difficulty $b^{(\text{specified})}=-2.0, 0.0, 2.0$.
  • ...and 1 more figures