CryptoLLM: Unleashing the Power of Prompted LLMs for SmartQnA and Classification of Crypto Posts
Aniket Deroy, Subhankar Maity
TL;DR
The paper addresses the problem of classifying cryptocurrency-related social media posts and extracting relevant answers to user questions. It proposes a prompt-based approach using GPT-4-Turbo and a 64-shot regime to handle both eight-way post classification and relevance detection in a crypto domain. Experiments on Reddit, Twitter, and a QnA dataset reveal relatively low macro-F1 scores for classification (0.249 for Reddit, 0.266 for Twitter) but a top rank in the QnA task (0.157), underscoring the difficulty of crypto-language classification while highlighting the potential of prompt engineering for domain-specific information retrieval. Overall, the work demonstrates the viability of prompting and few-shot learning for rapid prototyping and adaptation to evolving crypto-discourse tasks, with implications for market insight and automated discourse filtering.
Abstract
The rapid growth of social media has resulted in an large volume of user-generated content, particularly in niche domains such as cryptocurrency. This task focuses on developing robust classification models to accurately categorize cryptocurrency-related social media posts into predefined classes, including but not limited to objective, positive, negative, etc. Additionally, the task requires participants to identify the most relevant answers from a set of posts in response to specific questions. By leveraging advanced LLMs, this research aims to enhance the understanding and filtering of cryptocurrency discourse, thereby facilitating more informed decision-making in this volatile sector. We have used a prompt-based technique to solve the classification task for reddit posts and twitter posts. Also, we have used 64-shot technique along with prompts on GPT-4-Turbo model to determine whether a answer is relevant to a question or not.