Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Sentiment Classification and Irony Detection in Large Language Models through Advanced Prompt Engineering Techniques

Marvin Schmitt, Anne Schwerk, Sebastian Lempert

TL;DR

This paper investigates how advanced prompt engineering techniques can boost sentiment analysis performance in large language models, focusing on GPT-4o-mini and gemini-1.5-flash across binary and multilingual sentiment tasks, ABSA, and irony detection. It systematically compares baseline zero-shot prompts with few-shot, chain-of-thought, and self-consistency prompting, using 1000-sample evaluations and bootstrap-based significance testing. The main finding is that few-shot prompting yields robust improvements for GPT-4o-mini while chain-of-thought prompting dramatically enhances irony detection for gemini-1.5-flash, highlighting strong model-task dependencies. The work provides practical guidance on tailoring prompts to model architecture and task complexity, underscoring the need for careful calibration of reasoning-promoting prompts in affective computing applications. Overall, the study advances understanding of prompt design as a tool for optimizing sentiment analysis with minimal or no task-specific fine-tuning, with implications for real-world deployment in customer feedback, social monitoring, and related domains.

Abstract

This study investigates the use of prompt engineering to enhance large language models (LLMs), specifically GPT-4o-mini and gemini-1.5-flash, in sentiment analysis tasks. It evaluates advanced prompting techniques like few-shot learning, chain-of-thought prompting, and self-consistency against a baseline. Key tasks include sentiment classification, aspect-based sentiment analysis, and detecting subtle nuances such as irony. The research details the theoretical background, datasets, and methods used, assessing performance of LLMs as measured by accuracy, recall, precision, and F1 score. Findings reveal that advanced prompting significantly improves sentiment analysis, with the few-shot approach excelling in GPT-4o-mini and chain-of-thought prompting boosting irony detection in gemini-1.5-flash by up to 46%. Thus, while advanced prompting techniques overall improve performance, the fact that few-shot prompting works best for GPT-4o-mini and chain-of-thought excels in gemini-1.5-flash for irony detection suggests that prompting strategies must be tailored to both the model and the task. This highlights the importance of aligning prompt design with both the LLM's architecture and the semantic complexity of the task.

Enhancing Sentiment Classification and Irony Detection in Large Language Models through Advanced Prompt Engineering Techniques

TL;DR

This paper investigates how advanced prompt engineering techniques can boost sentiment analysis performance in large language models, focusing on GPT-4o-mini and gemini-1.5-flash across binary and multilingual sentiment tasks, ABSA, and irony detection. It systematically compares baseline zero-shot prompts with few-shot, chain-of-thought, and self-consistency prompting, using 1000-sample evaluations and bootstrap-based significance testing. The main finding is that few-shot prompting yields robust improvements for GPT-4o-mini while chain-of-thought prompting dramatically enhances irony detection for gemini-1.5-flash, highlighting strong model-task dependencies. The work provides practical guidance on tailoring prompts to model architecture and task complexity, underscoring the need for careful calibration of reasoning-promoting prompts in affective computing applications. Overall, the study advances understanding of prompt design as a tool for optimizing sentiment analysis with minimal or no task-specific fine-tuning, with implications for real-world deployment in customer feedback, social monitoring, and related domains.

Abstract

This study investigates the use of prompt engineering to enhance large language models (LLMs), specifically GPT-4o-mini and gemini-1.5-flash, in sentiment analysis tasks. It evaluates advanced prompting techniques like few-shot learning, chain-of-thought prompting, and self-consistency against a baseline. Key tasks include sentiment classification, aspect-based sentiment analysis, and detecting subtle nuances such as irony. The research details the theoretical background, datasets, and methods used, assessing performance of LLMs as measured by accuracy, recall, precision, and F1 score. Findings reveal that advanced prompting significantly improves sentiment analysis, with the few-shot approach excelling in GPT-4o-mini and chain-of-thought prompting boosting irony detection in gemini-1.5-flash by up to 46%. Thus, while advanced prompting techniques overall improve performance, the fact that few-shot prompting works best for GPT-4o-mini and chain-of-thought excels in gemini-1.5-flash for irony detection suggests that prompting strategies must be tailored to both the model and the task. This highlights the importance of aligning prompt design with both the LLM's architecture and the semantic complexity of the task.
Paper Structure (45 sections, 4 figures, 13 tables)

This paper contains 45 sections, 4 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. Research question
  4. Related work
  5. LLMs in sentiment analysis
  6. Prompt engineering and affective computing
  7. Summary of related work and positioning of this work
  8. Background
  9. Prompting
  10. Prompt engineering
  11. Sentiment analysis
  12. Experimental design
  13. Prompt engineering approaches
  14. Hypotheses and experimental setup
  15. Models
  16. ...and 30 more sections

Figures (4)

  • Figure D.1: Confusion matrix baseline vs. one-shot irony detection (gemini-flash1.5).
  • Figure D.2: Confusion matrix baseline vs. one-shot SB10k (gemini-flash-1.5).
  • Figure D.3: Confusion matrix baseline vs. CoT irony detection (gemini-flash-1.5)
  • Figure D.4: Confusion matrix few-shot vs. self-consistency SST2 (GPT-4o-mini).