Evaluation of NMT-Assisted Grammar Transfer for a Multi-Language Configurable Data-to-Text System
Andreas Madsack, Johanna Heininger, Adela Schneider, Ching-Yi Chen, Christian Eckard, Robert Weißgraeber
TL;DR
This paper presents a hybrid data-to-text system that leverages rule-based grammar configurations with neural machine translation to transfer grammatical units across languages, aiming to reduce hallucinations and scale multilingual generation. The approach uses a grammar-unit container, spaCy-based parsing, and a post-editing evaluation to quantify grammar-transfer quality, validated on the Sport-Sett:Basketball dataset. Results show high compatibility between automatic grammar transfer and human edits (98% unit-match), with domain-specific adjustments (notably German case handling) and some language-specific limitations. The study demonstrates the practicality of NMT-assisted grammar transfer within a configurable, rule-based NLG pipeline, while identifying areas for improvement and broader evaluation in future work.
Abstract
One approach for multilingual data-to-text generation is to translate grammatical configurations upfront from the source language into each target language. These configurations are then used by a surface realizer and in document planning stages to generate output. In this paper, we describe a rule-based NLG implementation of this approach where the configuration is translated by Neural Machine Translation (NMT) combined with a one-time human review, and introduce a cross-language grammar dependency model to create a multilingual NLG system that generates text from the source data, scaling the generation phase without a human in the loop. Additionally, we introduce a method for human post-editing evaluation on the automatically translated text. Our evaluation on the SportSett:Basketball dataset shows that our NLG system performs well, underlining its grammatical correctness in translation tasks.