Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OOVs in the Spotlight: How to Inflect them?

Tomáš Sourada, Jana Straková, Rudolf Rosa

TL;DR

This work tackles morphological inflection for out-of-vocabulary (OOV) words by evaluating three data-driven approaches—retrograde, LSTM-based seq2seq, and Transformer—on a Czech OOV inflection dataset and a manually annotated neologism set. It introduces the Czech OOV Inflection Dataset built from lemma-disjoint MorfFlexCZ20 data and a real-world OOV neologism test, enabling rigorous OOV evaluation. Across standard OOV conditions, Transformer performs best, while the retrograde method excels on neologisms; seq2seq models achieve state-of-the-art results in 9 of 16 languages in the SIGMORPHON 2022 OOV evaluation, highlighting cross-language robustness. The authors release the Czech OOV Inflection Dataset and a ready-to-use Python inflection library, contributing valuable benchmarks and tools for evaluating OOV inflection systems in morphologically rich languages.

Abstract

We focus on morphological inflection in out-of-vocabulary (OOV) conditions, an under-researched subtask in which state-of-the-art systems usually are less effective. We developed three systems: a retrograde model and two sequence-to-sequence (seq2seq) models based on LSTM and Transformer. For testing in OOV conditions, we automatically extracted a large dataset of nouns in the morphologically rich Czech language, with lemma-disjoint data splits, and we further manually annotated a real-world OOV dataset of neologisms. In the standard OOV conditions, Transformer achieves the best results, with increasing performance in ensemble with LSTM, the retrograde model and SIGMORPHON baselines. On the real-world OOV dataset of neologisms, the retrograde model outperforms all neural models. Finally, our seq2seq models achieve state-of-the-art results in 9 out of 16 languages from SIGMORPHON 2022 shared task data in the OOV evaluation (feature overlap) in the large data condition. We release the Czech OOV Inflection Dataset for rigorous evaluation in OOV conditions. Further, we release the inflection system with the seq2seq models as a ready-to-use Python library.

OOVs in the Spotlight: How to Inflect them?

TL;DR

This work tackles morphological inflection for out-of-vocabulary (OOV) words by evaluating three data-driven approaches—retrograde, LSTM-based seq2seq, and Transformer—on a Czech OOV inflection dataset and a manually annotated neologism set. It introduces the Czech OOV Inflection Dataset built from lemma-disjoint MorfFlexCZ20 data and a real-world OOV neologism test, enabling rigorous OOV evaluation. Across standard OOV conditions, Transformer performs best, while the retrograde method excels on neologisms; seq2seq models achieve state-of-the-art results in 9 of 16 languages in the SIGMORPHON 2022 OOV evaluation, highlighting cross-language robustness. The authors release the Czech OOV Inflection Dataset and a ready-to-use Python inflection library, contributing valuable benchmarks and tools for evaluating OOV inflection systems in morphologically rich languages.

Abstract

We focus on morphological inflection in out-of-vocabulary (OOV) conditions, an under-researched subtask in which state-of-the-art systems usually are less effective. We developed three systems: a retrograde model and two sequence-to-sequence (seq2seq) models based on LSTM and Transformer. For testing in OOV conditions, we automatically extracted a large dataset of nouns in the morphologically rich Czech language, with lemma-disjoint data splits, and we further manually annotated a real-world OOV dataset of neologisms. In the standard OOV conditions, Transformer achieves the best results, with increasing performance in ensemble with LSTM, the retrograde model and SIGMORPHON baselines. On the real-world OOV dataset of neologisms, the retrograde model outperforms all neural models. Finally, our seq2seq models achieve state-of-the-art results in 9 out of 16 languages from SIGMORPHON 2022 shared task data in the OOV evaluation (feature overlap) in the large data condition. We release the Czech OOV Inflection Dataset for rigorous evaluation in OOV conditions. Further, we release the inflection system with the seq2seq models as a ready-to-use Python library.
Paper Structure (31 sections, 2 equations, 3 figures, 7 tables)

This paper contains 31 sections, 2 equations, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Czech OOV Inflection Dataset
  4. MorfFlex Morphological Dictionary
  5. Neologisms
  6. Methods
  7. Evaluation Metrics
  8. Form accuracy
  9. Full-paradigm accuracy
  10. Baseline Systems
  11. copy
  12. sklonuj.cz
  13. SIG nonneur
  14. SIG trm
  15. SIG trm-tune
  16. ...and 16 more sections

Figures (3)

  • Figure 1: Retrograde model development evaluation with different sizes of training dataset. Train size = count of paradigm tables in the training dataset.
  • Figure 2: Input-output example for seq2seq models, and comparison of inflection task and the MT task. had (snake) is the lemma, "#" is a separator, P6 is the morphological tag describing the target form (plural, locative case), hadech (snakes used as in about snakes) is the target inflected form.
  • Figure 3: Histogram: the percentage of paradigm tables with given number of errors, counted for all incorrectly generated paradigm tables, for each model separately.