The Overlooked Repetitive Lengthening Form in Sentiment Analysis

Abstract

Individuals engaging in online communication frequently express personal opinions with informal styles (e.g., memes and emojis). While Language Models (LMs) with informal communications have been widely discussed, a unique and emphatic style, the Repetitive Lengthening Form (RLF), has been overlooked for years. In this paper, we explore answers to two research questions: 1) Is RLF important for sentiment analysis (SA)? 2) Can LMs understand RLF? Inspired by previous linguistic research, we curate \textbf{Lengthening}, the first multi-domain dataset with 850k samples focused on RLF for SA. Moreover, we introduce \textbf{Exp}lainable \textbf{Instruct}ion Tuning (\textbf{ExpInstruct}), a two-stage instruction tuning framework aimed to improve both performance and explainability of LLMs for RLF. We further propose a novel unified approach to quantify LMs' understanding of informal expressions. We show that RLF sentences are expressive expressions and can serve as signatures of document-level sentiment. Additionally, RLF has potential value for online content analysis. Our results show that fine-tuned Pre-trained Language Models (PLMs) can surpass zero-shot GPT-4 in performance but not in explanation for RLF. Finally, we show ExpInstruct can improve the open-sourced LLMs to match zero-shot GPT-4 in performance and explainability for RLF with limited samples. Code and sample data are available at https://github.com/Tom-Owl/OverlookedRLF

Figures (5)

• Figure 1: An overview of our work for RLF. (1) We introduce Lengthening in Section \ref{['sec:Lengthening_dataset']}. (2) We propose the ExpInstruct framework and describe prompt details in Section \ref{['sec:expInstruct']}. (3) Experiments details are in Section \ref{['sec:exp_setup']}.
• Figure 2: Prompt Design and Template for ExpInstruct. (a) Prompt with CoT for word-level explainability. (b) Simple Prompt for SA. (c) Prompt Template for Instruction tuning
• Figure 3: Comparing normalized WIS for an RLF sentence from zero-shot GPT-4 and fine-tuned RoBERTa.
• Figure 4: Comparison of accuracy between the RLF and w/o RLF groups using zero-shot and fine-tuned models by sentence length. The lines represent average values, and the error bar indicate the standard deviation for each length group across 3 runs. Both results across 3 fine-tuned models show a convergence between the RLF and w/o RLF groups when the sentence character length is around 80.
• Figure 5: Our customized user interface for human evaluation. Annotators are asked to do two tasks: annotation for sentiment labels and explanation reliability.