Creating a Hybrid Rule and Neural Network Based Semantic Tagger using Silver Standard Data: the PyMUSAS framework for Multilingual Semantic Annotation

TL;DR

This work tackles the lack of extensive evaluation for USAS tagging across multiple languages by constructing a silver-labelled English dataset and assembling evaluation data in five languages, including a newly created Chinese corpus. It demonstrates that neural and hybrid approaches, built atop the PyMUSAS rule-based framework, outperform traditional rule-based tagging in multilingual semantic tagging, with cross-lingual transfer benefiting from large pre-training corpora. The authors release open resources, including the Chinese corpus and silver English data, and show that a hybrid rule–neural framework provides robust coverage and accuracy. Overall, the study advances multilingual semantic annotation under the USAS framework and highlights practical pathways for scalable cross-lingual semantic tagging.

Abstract

Word Sense Disambiguation (WSD) has been widely evaluated using the semantic frameworks of WordNet, BabelNet, and the Oxford Dictionary of English. However, for the UCREL Semantic Analysis System (USAS) framework, no open extensive evaluation has been performed beyond lexical coverage or single language evaluation. In this work, we perform the largest semantic tagging evaluation of the rule based system that uses the lexical resources in the USAS framework covering five different languages using four existing datasets and one novel Chinese dataset. We create a new silver labelled English dataset, to overcome the lack of manually tagged training data, that we train and evaluate various mono and multilingual neural models in both mono and cross-lingual evaluation setups with comparisons to their rule based counterparts, and show how a rule based system can be enhanced with a neural network model. The resulting neural network models, including the data they were trained on, the Chinese evaluation dataset, and all of the code have been released as open resources.

Figures (2)

• Figure 1: Architecture of the WSD Bi-Encoder Model (BEM) from blevins-zettlemoyer-2020-moving.
• Figure 2: Probability of a USAS label within the silver labelled training data, training split, for each distribution.