STF: Sentence Transformer Fine-Tuning For Topic Categorization With Limited Data
Kheir Eddine Daouadi, Yaakoub Boualleg, Oussama Guehairia
TL;DR
This paper tackles tweet topic classification under limited labeled data by introducing STF: Sentence Transformer Fine-tuning for Topic Categorization With Limited Data. STF leverages pretrained sentence-transformer models in an end-to-end fine-tuning framework, using transfer learning to capture long-range dependencies without requiring large labeled corpora. Across two English tweet datasets, STF—especially with all-roberta-large-v1 and all-mpnet-base-v1 configurations—outperforms state-of-the-art baselines, achieving 91.63% accuracy on one dataset and 77.52% on the other, demonstrating strong data efficiency. The work highlights a practical, scalable approach for social-media topic detection and points to future avenues in vocabulary customization and data augmentation for domain-adapted sentence transformers.
Abstract
Nowadays, topic classification from tweets attracts considerable research attention. Different classification systems have been suggested thanks to these research efforts. Nevertheless, they face major challenges owing to low performance metrics due to the limited amount of labeled data. We propose Sentence Transformers Fine-tuning (STF), a topic detection system that leverages pretrained Sentence Transformers models and fine-tuning to classify topics from tweets accurately. Moreover, extensive parameter sensitivity analyses were conducted to finetune STF parameters for our topic classification task to achieve the best performance results. Experiments on two benchmark datasets demonstrated that (1) the proposed STF can be effectively used for classifying tweet topics and outperforms the latest state-of-the-art approaches, and (2) the proposed STF does not require a huge amount of labeled tweets to achieve good accuracy, which is a limitation of many state-of-the-art approaches. Our main contribution is the achievement of promising results in tweet topic classification by applying pretrained sentence transformers language models.