Unlocking Cross-Lingual Sentiment Analysis through Emoji Interpretation: A Multimodal Generative AI Approach
Rafid Ishrak Jahan, Heng Fan, Haihua Chen, Yunhe Feng
TL;DR
This study investigates whether emojis can serve as universal sentiment indicators across languages and cultures. It builds a large, multimodal emoji representation dataset (icons, titles, descriptions, and pixels) for 5030 emojis and uses GPT-4o to evaluate which representations best indicate sentiment, benchmarking against ESR v1.0. It develops three standalone sentiment algorithms (BSA, DPM, and Majority Voting) and a position-aware refinement, showing that emoji-based sentiment can reach up to $81.43\%$ accuracy when prioritizing the first emoji and using multimodal representations. The multilingual evaluation spans 19 languages across 32 countries, using World Cup-related tweets; results demonstrate the potential of emojis as universal sentiment markers and inform cross-lingual sentiment analysis on social media. Code and resources are provided to support reproducibility and further research.
Abstract
Emojis have become ubiquitous in online communication, serving as a universal medium to convey emotions and decorative elements. Their widespread use transcends language and cultural barriers, enhancing understanding and fostering more inclusive interactions. While existing work gained valuable insight into emojis understanding, exploring emojis' capability to serve as a universal sentiment indicator leveraging large language models (LLMs) has not been thoroughly examined. Our study aims to investigate the capacity of emojis to serve as reliable sentiment markers through LLMs across languages and cultures. We leveraged the multimodal capabilities of ChatGPT to explore the sentiments of various representations of emojis and evaluated how well emoji-conveyed sentiment aligned with text sentiment on a multi-lingual dataset collected from 32 countries. Our analysis reveals that the accuracy of LLM-based emoji-conveyed sentiment is 81.43%, underscoring emojis' significant potential to serve as a universal sentiment marker. We also found a consistent trend that the accuracy of sentiment conveyed by emojis increased as the number of emojis grew in text. The results reinforce the potential of emojis to serve as global sentiment indicators, offering insight into fields such as cross-lingual and cross-cultural sentiment analysis on social media platforms. Code: https://github.com/ResponsibleAILab/emoji-universal-sentiment.