dzStance at StanceEval2024: Arabic Stance Detection based on Sentence Transformers

TL;DR

This work assesses Arabic stance detection by comparing TF-IDF features with Sentence Transformers on Mawqif 2022 data across three topics. It demonstrates that embedding-based representations (xlm-r-bert-base-nli-stsb-mean-tokens) with Logistic Regression outperform traditional TF-IDF with LinearSVC, achieving an overall F1 of 68.48% in the experiments and placing competitively in the StanceEval2024 rankings (e.g., 13th overall). The findings underscore the effectiveness of transformer-based embeddings for capturing semantic nuances in Arabic text and suggest avenues for further improvements via hybrid feature strategies and targeted fine-tuning. The work contributes practical, reproducible approaches for Arabic stance detection with potential impact on social discourse analysis and monitoring.

Abstract

This study compares Term Frequency-Inverse Document Frequency (TF-IDF) features with Sentence Transformers for detecting writers' stances--favorable, opposing, or neutral--towards three significant topics: COVID-19 vaccine, digital transformation, and women empowerment. Through empirical evaluation, we demonstrate that Sentence Transformers outperform TF-IDF features across various experimental setups. Our team, dzStance, participated in a stance detection competition, achieving the 13th position (74.91%) among 15 teams in Women Empowerment, 10th (73.43%) in COVID Vaccine, and 12th (66.97%) in Digital Transformation. Overall, our team's performance ranked 13th (71.77%) among all participants. Notably, our approach achieved promising F1-scores, highlighting its effectiveness in identifying writers' stances on diverse topics. These results underscore the potential of Sentence Transformers to enhance stance detection models for addressing critical societal issues.