Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

To Drop or Not to Drop? Predicting Argument Ellipsis Judgments: A Case Study in Japanese

Yukiko Ishizuki, Tatsuki Kuribayashi, Yuichiroh Matsubayashi, Ryohei Sasano, Kentaro Inui

Abstract

Speakers sometimes omit certain arguments of a predicate in a sentence; such omission is especially frequent in pro-drop languages. This study addresses a question about ellipsis -- what can explain the native speakers' ellipsis decisions? -- motivated by the interest in human discourse processing and writing assistance for this choice. To this end, we first collect large-scale human annotations of whether and why a particular argument should be omitted across over 2,000 data points in the balanced corpus of Japanese, a prototypical pro-drop language. The data indicate that native speakers overall share common criteria for such judgments and further clarify their quantitative characteristics, e.g., the distribution of related linguistic factors in the balanced corpus. Furthermore, the performance of the language model-based argument ellipsis judgment model is examined, and the gap between the systems' prediction and human judgments in specific linguistic aspects is revealed. We hope our fundamental resource encourages further studies on natural human ellipsis judgment.

To Drop or Not to Drop? Predicting Argument Ellipsis Judgments: A Case Study in Japanese

Abstract

Speakers sometimes omit certain arguments of a predicate in a sentence; such omission is especially frequent in pro-drop languages. This study addresses a question about ellipsis -- what can explain the native speakers' ellipsis decisions? -- motivated by the interest in human discourse processing and writing assistance for this choice. To this end, we first collect large-scale human annotations of whether and why a particular argument should be omitted across over 2,000 data points in the balanced corpus of Japanese, a prototypical pro-drop language. The data indicate that native speakers overall share common criteria for such judgments and further clarify their quantitative characteristics, e.g., the distribution of related linguistic factors in the balanced corpus. Furthermore, the performance of the language model-based argument ellipsis judgment model is examined, and the gap between the systems' prediction and human judgments in specific linguistic aspects is revealed. We hope our fundamental resource encourages further studies on natural human ellipsis judgment.
Paper Structure (56 sections, 5 figures, 8 tables)

This paper contains 56 sections, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Japanese Argument Ellipsis
  3. Annotation Task Design
  4. Annotation Overview:
  5. Source Texts
  6. Corpus:
  7. Omission Statistics:
  8. Data Presented to Workers:
  9. Preprocess: Argument Insertion
  10. Factor-level Questions
  11. Factors:
  12. Constraints and Preferences:
  13. From Factor-level Answers to Ellipsis Judgments
  14. Annotation Interface
  15. Interface:
  16. ...and 41 more sections

Figures (5)

  • Figure 1: Illustration of the argument ellipsis judgment annotation. In addition to asking for the final judgments, we also asked workers to answer the questions corresponding to the linguistic factors that potentially influence their judgments.
  • Figure 2: Annotation procedure for our ellipsis judgment task.
  • Figure 3: The decision tree used in the annotation process. Each number denotes the factor ID. The validity of the tree was confirmed during the preliminary tasks among the authors and the instructions provided to the workers.
  • Figure 4: Confusion matrix of humans (left side) and BERT-large (right side) in validation data. Note that the numbers of instances are based on the average of one-vs-one evaluations for humans and based on the comparison between the system's predictions and the aggregated labels of humans for BERT$_\mathrm{L}$.
  • Figure 5: Prompt example for the GPT models. The targeted predicate and argument are surrounded by special tags (<argument> or <predicate>). An explanation of the targeted factor and an example illustrated in the annotation manual are also provided to the model.