Normalization of Lithuanian Text Using Regular Expressions

TL;DR

This work develops a regex-driven framework for Lithuanian text normalization in TTS by defining a taxonomy of non-standard words, assembling a large rule set, and evaluating on three diverse datasets. The approach emphasizes long-context rules, tagging for inflection, and targeted handling of cardinal/ordinal numerals, dates, years, times, and contact information. Experiments show that splitting data into rulemaking and testing parts reduces errors, with NAV-generated data yielding the best results and a notable share of errors linked to abbreviations and inflection decisions. The findings highlight practical guidance for expanding NSW rules, recommending restrained expansion (notably for units of measure) to improve robustness across domains.

Abstract

Text Normalization is an integral part of any text-to-speech synthesis system. In a natural language text, there are elements such as numbers, dates, abbreviations, etc. that belong to other semiotic classes. They are called non-standard words (NSW) and need to be expanded into ordinary words. For this purpose, it is necessary to identify the semiotic class of each NSW. The taxonomy of semiotic classes adapted to the Lithuanian language is presented in the work. Sets of rules are created for detecting and expanding NSWs based on regular expressions. Experiments with three completely different data sets were performed and the accuracy was assessed. Causes of errors are explained and recommendations are given for the development of text normalization rules.