NJUST-KMG at TRAC-2024 Tasks 1 and 2: Offline Harm Potential Identification

TL;DR

This work tackles offline harm potential identification from multilingual Indian-language social media, framing it as a 4-class classification for harm potential (sub-task 1a) with a separate sub-task for target identification (1b). The authors fine-tune pretrained multilingual transformers (XLM-R, MuRILBERT, BanglaBERT) and enhance discriminative power via contrastive learning using the infoNCE objective $L_{infoNCE} = -\frac{1}{N} \sum_{i=1}^N \log \frac{\exp(sim(z_i, z_{i+}) / \tau)}{\sum_{j=1}^N \exp(sim(z_i, z_{i,j}) / \tau)}$, followed by an ensemble of models to improve generalization. Ablation studies show that contrastive loss and ensemble methods contribute to higher F1 scores, with the best ensemble achieving $F1 = 0.73$ on sub-task 1a; larger models did not always yield gains. The results demonstrate a robust, multilingual approach capable of handling cross-lingual nuances in harm-potential detection, offering a practical pathway for safer moderation in multilingual online spaces.

Abstract

This report provide a detailed description of the method that we proposed in the TRAC-2024 Offline Harm Potential dentification which encloses two sub-tasks. The investigation utilized a rich dataset comprised of social media comments in several Indian languages, annotated with precision by expert judges to capture the nuanced implications for offline context harm. The objective assigned to the participants was to design algorithms capable of accurately assessing the likelihood of harm in given situations and identifying the most likely target(s) of offline harm. Our approach ranked second in two separate tracks, with F1 values of 0.73 and 0.96 respectively. Our method principally involved selecting pretrained models for finetuning, incorporating contrastive learning techniques, and culminating in an ensemble approach for the test set.