Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Flexing in 73 Languages: A Single Small Model for Multilingual Inflection

Tomáš Sourada, Jana Straková

TL;DR

This work tackles multilingual morphological inflection by training a single compact Transformer model on $73$ languages, achieving robust performance on unseen words with a lightweight $5.69\mathrm{M}$ parameter footprint. A language ID token enables true multilingual inference, while a frequency-weighted, lemma-disjoint data splitting strategy and temperature-controlled upsampling balance data across languages. The multilingual model attains strong results on SIGMORPHON benchmarks (ranking $3^{rd}$ in 2022 and $1^{st}$ in 2023) and provides state-of-the-art performance on Universal Dependencies across 73 languages, with a macro-average improvement of $4.69\%$ over monolingual baselines. The approach emphasizes deployment practicality, offering an open-source solution that eliminates the need to maintain dozens of monolingual models, while still handling OOV forms for diverse linguistic families.

Abstract

We present a compact, single-model approach to multilingual inflection, the task of generating inflected word forms from base lemmas to express grammatical categories. Our model, trained jointly on data from 73 languages, is lightweight, robust to unseen words, and outperforms monolingual baselines in most languages. This demonstrates the effectiveness of multilingual modeling for inflection and highlights its practical benefits: simplifying deployment by eliminating the need to manage and retrain dozens of separate monolingual models. In addition to the standard SIGMORPHON shared task benchmarks, we evaluate our monolingual and multilingual models on 73 Universal Dependencies (UD) treebanks, extracting lemma-tag-form triples and their frequency counts. To ensure realistic data splits, we introduce a novel frequency-weighted, lemma-disjoint train-dev-test resampling procedure. Our work addresses the lack of an open-source, general-purpose, multilingual morphological inflection system capable of handling unseen words across a wide range of languages, including Czech. All code is publicly released at: https://github.com/tomsouri/multilingual-inflection.

Flexing in 73 Languages: A Single Small Model for Multilingual Inflection

TL;DR

This work tackles multilingual morphological inflection by training a single compact Transformer model on languages, achieving robust performance on unseen words with a lightweight parameter footprint. A language ID token enables true multilingual inference, while a frequency-weighted, lemma-disjoint data splitting strategy and temperature-controlled upsampling balance data across languages. The multilingual model attains strong results on SIGMORPHON benchmarks (ranking in 2022 and in 2023) and provides state-of-the-art performance on Universal Dependencies across 73 languages, with a macro-average improvement of over monolingual baselines. The approach emphasizes deployment practicality, offering an open-source solution that eliminates the need to maintain dozens of monolingual models, while still handling OOV forms for diverse linguistic families.

Abstract

We present a compact, single-model approach to multilingual inflection, the task of generating inflected word forms from base lemmas to express grammatical categories. Our model, trained jointly on data from 73 languages, is lightweight, robust to unseen words, and outperforms monolingual baselines in most languages. This demonstrates the effectiveness of multilingual modeling for inflection and highlights its practical benefits: simplifying deployment by eliminating the need to manage and retrain dozens of separate monolingual models. In addition to the standard SIGMORPHON shared task benchmarks, we evaluate our monolingual and multilingual models on 73 Universal Dependencies (UD) treebanks, extracting lemma-tag-form triples and their frequency counts. To ensure realistic data splits, we introduce a novel frequency-weighted, lemma-disjoint train-dev-test resampling procedure. Our work addresses the lack of an open-source, general-purpose, multilingual morphological inflection system capable of handling unseen words across a wide range of languages, including Czech. All code is publicly released at: https://github.com/tomsouri/multilingual-inflection.
Paper Structure (13 sections, 1 figure, 4 tables)

This paper contains 13 sections, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data
  4. Methodology
  5. Data Splitting
  6. Model
  7. Multilingual Model
  8. Results
  9. SIGMORPHON Benchmarks
  10. Universal Dependencies
  11. Conclusions
  13. Appendix: Hyperparameters

Figures (1)

  • Figure 1: The morphological inflection task: an input-output example, inflection of English lemma "well" to superlative, inflected form "best".