Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Clue-Instruct: Text-Based Clue Generation for Educational Crossword Puzzles

Andrea Zugarini, Kamyar Zeinalipour, Surya Sai Kadali, Marco Maggini, Marco Gori, Leonardo Rigutini

TL;DR

This work tackles the scarcity of educational crossword clues grounded in explicit source content. It introduces clue-instruct, a 44,075-example dataset built from Wikipedia contexts and three pedagogical clues per entry, generated by LLMs and grounded to textual passages to reduce hallucinations. The authors demonstrate that instruction-tuning open-source LLMs on clue-instruct substantially improves clue quality, both automatically (ROUGE-L) and in human evaluations, with larger models and fine-tuning delivering the strongest results. The dataset and trained models are publicly released, enabling educational crossword generation and paving the way for multilingual extensions in future work.

Abstract

Crossword puzzles are popular linguistic games often used as tools to engage students in learning. Educational crosswords are characterized by less cryptic and more factual clues that distinguish them from traditional crossword puzzles. Despite there exist several publicly available clue-answer pair databases for traditional crosswords, educational clue-answer pairs datasets are missing. In this article, we propose a methodology to build educational clue generation datasets that can be used to instruct Large Language Models (LLMs). By gathering from Wikipedia pages informative content associated with relevant keywords, we use Large Language Models to automatically generate pedagogical clues related to the given input keyword and its context. With such an approach, we created clue-instruct, a dataset containing 44,075 unique examples with text-keyword pairs associated with three distinct crossword clues. We used clue-instruct to instruct different LLMs to generate educational clues from a given input content and keyword. Both human and automatic evaluations confirmed the quality of the generated clues, thus validating the effectiveness of our approach.

Clue-Instruct: Text-Based Clue Generation for Educational Crossword Puzzles

TL;DR

This work tackles the scarcity of educational crossword clues grounded in explicit source content. It introduces clue-instruct, a 44,075-example dataset built from Wikipedia contexts and three pedagogical clues per entry, generated by LLMs and grounded to textual passages to reduce hallucinations. The authors demonstrate that instruction-tuning open-source LLMs on clue-instruct substantially improves clue quality, both automatically (ROUGE-L) and in human evaluations, with larger models and fine-tuning delivering the strongest results. The dataset and trained models are publicly released, enabling educational crossword generation and paving the way for multilingual extensions in future work.

Abstract

Crossword puzzles are popular linguistic games often used as tools to engage students in learning. Educational crosswords are characterized by less cryptic and more factual clues that distinguish them from traditional crossword puzzles. Despite there exist several publicly available clue-answer pair databases for traditional crosswords, educational clue-answer pairs datasets are missing. In this article, we propose a methodology to build educational clue generation datasets that can be used to instruct Large Language Models (LLMs). By gathering from Wikipedia pages informative content associated with relevant keywords, we use Large Language Models to automatically generate pedagogical clues related to the given input keyword and its context. With such an approach, we created clue-instruct, a dataset containing 44,075 unique examples with text-keyword pairs associated with three distinct crossword clues. We used clue-instruct to instruct different LLMs to generate educational clues from a given input content and keyword. Both human and automatic evaluations confirmed the quality of the generated clues, thus validating the effectiveness of our approach.
Paper Structure (29 sections, 10 figures, 3 tables)

This paper contains 29 sections, 10 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Crossword solving.
  4. Crossword Generation.
  5. Clue-Answer datasets.
  6. Method
  7. Data Retrieval.
  8. Data Screening.
  9. Craft the prompt.
  10. Clues Generation.
  11. Clue-Instruct Dataset
  12. Statistics
  13. Measuring Data Quality
  14. Automatic Metrics.
  15. Human Evaluation.
  16. ...and 14 more sections

Figures (10)

  • Figure 1: Example of an educational crossword puzzle on Geography-related keywords.
  • Figure 2: clue-instruct prompt used to generate the clues.
  • Figure 3: This figure illustrates the pipeline employed in constructing clue-instruct: (a) Information extraction from Wikipedia pages of text, keywords, and categories. (b) Data refinement and filtering to enhance data quality, by selecting the most crucial and highly-viewed pages, eliminating excessively short or overly detailed text, and more. (c) Design of the prompt for generating crossword clues based on input text and specified keywords within specific categories. (d) Exploit of GPT-3.5-Turbo to generate clues from the collected data and defined prompts.
  • Figure 4: Distribution of the examples among the twenty categories.
  • Figure 5: Word length distribution of contexts and outputs. Char length distribution over keywords.
  • ...and 5 more figures