Universal Cross-Lingual Text Classification
Riya Savant, Anushka Shelke, Sakshi Todmal, Sanskruti Kanphade, Ananya Joshi, Raviraj Joshi
TL;DR
The paper tackles the scarcity of labeled data in low-resource languages for text classification by proposing universal cross-lingual text classification, which merges supervised labels from multiple languages to expand label coverage. It builds on multilingual SBERT baselines (IndicSBERT, LaBSE, LASER) and evaluates cross-lingual transfer and label-union training on the IndicNLP dataset. The results show that IndicSBERT often outperforms baselines, and that training with diverse language-label combinations enables classification in unseen languages, achieving high accuracy and broader label support. This approach has practical significance for deploying NLP tools in low-resource languages by enabling universal cross-lingual transfer with expanded label sets.
Abstract
Text classification, an integral task in natural language processing, involves the automatic categorization of text into predefined classes. Creating supervised labeled datasets for low-resource languages poses a considerable challenge. Unlocking the language potential of low-resource languages requires robust datasets with supervised labels. However, such datasets are scarce, and the label space is often limited. In our pursuit to address this gap, we aim to optimize existing labels/datasets in different languages. This research proposes a novel perspective on Universal Cross-Lingual Text Classification, leveraging a unified model across languages. Our approach involves blending supervised data from different languages during training to create a universal model. The supervised data for a target classification task might come from different languages covering different labels. The primary goal is to enhance label and language coverage, aiming for a label set that represents a union of labels from various languages. We propose the usage of a strong multilingual SBERT as our base model, making our novel training strategy feasible. This strategy contributes to the adaptability and effectiveness of the model in cross-lingual language transfer scenarios, where it can categorize text in languages not encountered during training. Thus, the paper delves into the intricacies of cross-lingual text classification, with a particular focus on its application for low-resource languages, exploring methodologies and implications for the development of a robust and adaptable universal cross-lingual model.