Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Dataset of Open-Domain Question Answering with Multiple-Span Answers

Zhiyi Luo, Yingying Zhang, Shuyun Luo, Ying Zhao, Wentao Lyu

TL;DR

This paper introduces CLEAN, a comprehensive open-domain Chinese multi-span question answering dataset designed to fill the gap in publicly available Chinese MSQA benchmarks. CLEAN emphasizes descriptive, long-span answers sourced from Baidu Zhidao, yielding 9,063 samples with nearly half being multi-span and a 76% descriptive answer rate, significantly challenging existing models. The authors establish strong baselines across extractive and generative approaches, noting that current methods struggle with CLEAN’s descriptive questions, and demonstrating that multi-span extraction in Chinese remains a harder problem than English counterparts. The dataset and baselines aim to catalyze progress in Chinese MSQA research and open-domain QA, with CLEAN slated for public release to the research community.

Abstract

Multi-span answer extraction, also known as the task of multi-span question answering (MSQA), is critical for real-world applications, as it requires extracting multiple pieces of information from a text to answer complex questions. Despite the active studies and rapid progress in English MSQA research, there is a notable lack of publicly available MSQA benchmark in Chinese. Previous efforts for constructing MSQA datasets predominantly emphasized entity-centric contextualization, resulting in a bias towards collecting factoid questions and potentially overlooking questions requiring more detailed descriptive responses. To overcome these limitations, we present CLEAN, a comprehensive Chinese multi-span question answering dataset that involves a wide range of open-domain subjects with a substantial number of instances requiring descriptive answers. Additionally, we provide established models from relevant literature as baselines for CLEAN. Experimental results and analysis show the characteristics and challenge of the newly proposed CLEAN dataset for the community. Our dataset, CLEAN, will be publicly released at zhiyiluo.site/misc/clean_v1.0_ sample.json.

A Dataset of Open-Domain Question Answering with Multiple-Span Answers

TL;DR

This paper introduces CLEAN, a comprehensive open-domain Chinese multi-span question answering dataset designed to fill the gap in publicly available Chinese MSQA benchmarks. CLEAN emphasizes descriptive, long-span answers sourced from Baidu Zhidao, yielding 9,063 samples with nearly half being multi-span and a 76% descriptive answer rate, significantly challenging existing models. The authors establish strong baselines across extractive and generative approaches, noting that current methods struggle with CLEAN’s descriptive questions, and demonstrating that multi-span extraction in Chinese remains a harder problem than English counterparts. The dataset and baselines aim to catalyze progress in Chinese MSQA research and open-domain QA, with CLEAN slated for public release to the research community.

Abstract

Multi-span answer extraction, also known as the task of multi-span question answering (MSQA), is critical for real-world applications, as it requires extracting multiple pieces of information from a text to answer complex questions. Despite the active studies and rapid progress in English MSQA research, there is a notable lack of publicly available MSQA benchmark in Chinese. Previous efforts for constructing MSQA datasets predominantly emphasized entity-centric contextualization, resulting in a bias towards collecting factoid questions and potentially overlooking questions requiring more detailed descriptive responses. To overcome these limitations, we present CLEAN, a comprehensive Chinese multi-span question answering dataset that involves a wide range of open-domain subjects with a substantial number of instances requiring descriptive answers. Additionally, we provide established models from relevant literature as baselines for CLEAN. Experimental results and analysis show the characteristics and challenge of the newly proposed CLEAN dataset for the community. Our dataset, CLEAN, will be publicly released at zhiyiluo.site/misc/clean_v1.0_ sample.json.
Paper Structure (12 sections, 1 figure, 4 tables)

This paper contains 12 sections, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Dataset Construction
  3. Data Collection
  4. Annotation Process
  5. Dataset Statistics
  6. Experiments
  7. Datasets
  8. Models
  9. Experimental Setup
  10. Experimental Results and Analysis
  11. Related Work
  12. Conclusion

Figures (1)

  • Figure 1: Subjects of questions in CLEAN.