AraFinNLP 2024: The First Arabic Financial NLP Shared Task

TL;DR

AraFinNLP 2024 tackles Arabic financial NLP across dialects with two subtasks: multi-dialect intent detection and cross-dialect translation preserving intent. The shared task extends ArBanking77 to include Palestinian, Moroccan, Saudi, and Tunisian dialects, and uses a JSON-Codalab submission format. The top result in Subtask 1 achieved a Micro-F1 of 0.8773 via an ensemble of fine-tuned BERT-based models with contrastive loss and added dialect data, while Subtask 2's sole entry reached BLEU 1.667, underscoring the challenges of cross-dialect fidelity. Overall, the work demonstrates the feasibility of dialect-aware financial NLP in Arabic and points to data augmentation and model ensembles as effective strategies, with future directions including broader dialect coverage and domain-specific resources.

Abstract

The expanding financial markets of the Arab world require sophisticated Arabic NLP tools. To address this need within the banking domain, the Arabic Financial NLP (AraFinNLP) shared task proposes two subtasks: (i) Multi-dialect Intent Detection and (ii) Cross-dialect Translation and Intent Preservation. This shared task uses the updated ArBanking77 dataset, which includes about 39k parallel queries in MSA and four dialects. Each query is labeled with one or more of a common 77 intents in the banking domain. These resources aim to foster the development of robust financial Arabic NLP, particularly in the areas of machine translation and banking chat-bots. A total of 45 unique teams registered for this shared task, with 11 of them actively participated in the test phase. Specifically, 11 teams participated in Subtask 1, while only 1 team participated in Subtask 2. The winning team of Subtask 1 achieved F1 score of 0.8773, and the only team submitted in Subtask 2 achieved a 1.667 BLEU score.

Figures (2)

• Figure 1: AraBanking2024 Datasets for Intent Detection
• Figure 2: Translation of MSA into four Arabic dialects.