Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deciphering Political Entity Sentiment in News with Large Language Models: Zero-Shot and Few-Shot Strategies

Alapan Kuila, Sudeshna Sarkar

TL;DR

The paper addresses entity-centric sentiment analysis in political news and evaluates large language models (LLMs) for predicting sentiment toward specific entities using zero-shot and few-shot prompting. It leverages chain-of-thought prompting with rationale and a self-consistency decoding strategy across multiple open LLMs, comparing against a fine-tuned BERT baseline on two datasets. Results show that LLMs, particularly Falcon-40B, can outperform FT-BERT in many settings, with few-shot learning and self-consistency boosting performance, while the effectiveness of chain-of-thought prompting is inconsistent across datasets. These findings demonstrate the viability of resource-efficient LLMs for entity-centric sentiment tasks in political news and provide practical guidance on prompting strategies and model choice for real-world political NLP applications.

Abstract

Sentiment analysis plays a pivotal role in understanding public opinion, particularly in the political domain where the portrayal of entities in news articles influences public perception. In this paper, we investigate the effectiveness of Large Language Models (LLMs) in predicting entity-specific sentiment from political news articles. Leveraging zero-shot and few-shot strategies, we explore the capability of LLMs to discern sentiment towards political entities in news content. Employing a chain-of-thought (COT) approach augmented with rationale in few-shot in-context learning, we assess whether this method enhances sentiment prediction accuracy. Our evaluation on sentiment-labeled datasets demonstrates that LLMs, outperform fine-tuned BERT models in capturing entity-specific sentiment. We find that learning in-context significantly improves model performance, while the self-consistency mechanism enhances consistency in sentiment prediction. Despite the promising results, we observe inconsistencies in the effectiveness of the COT prompting method. Overall, our findings underscore the potential of LLMs in entity-centric sentiment analysis within the political news domain and highlight the importance of suitable prompting strategies and model architectures.

Deciphering Political Entity Sentiment in News with Large Language Models: Zero-Shot and Few-Shot Strategies

TL;DR

The paper addresses entity-centric sentiment analysis in political news and evaluates large language models (LLMs) for predicting sentiment toward specific entities using zero-shot and few-shot prompting. It leverages chain-of-thought prompting with rationale and a self-consistency decoding strategy across multiple open LLMs, comparing against a fine-tuned BERT baseline on two datasets. Results show that LLMs, particularly Falcon-40B, can outperform FT-BERT in many settings, with few-shot learning and self-consistency boosting performance, while the effectiveness of chain-of-thought prompting is inconsistent across datasets. These findings demonstrate the viability of resource-efficient LLMs for entity-centric sentiment tasks in political news and provide practical guidance on prompting strategies and model choice for real-world political NLP applications.

Abstract

Sentiment analysis plays a pivotal role in understanding public opinion, particularly in the political domain where the portrayal of entities in news articles influences public perception. In this paper, we investigate the effectiveness of Large Language Models (LLMs) in predicting entity-specific sentiment from political news articles. Leveraging zero-shot and few-shot strategies, we explore the capability of LLMs to discern sentiment towards political entities in news content. Employing a chain-of-thought (COT) approach augmented with rationale in few-shot in-context learning, we assess whether this method enhances sentiment prediction accuracy. Our evaluation on sentiment-labeled datasets demonstrates that LLMs, outperform fine-tuned BERT models in capturing entity-specific sentiment. We find that learning in-context significantly improves model performance, while the self-consistency mechanism enhances consistency in sentiment prediction. Despite the promising results, we observe inconsistencies in the effectiveness of the COT prompting method. Overall, our findings underscore the potential of LLMs in entity-centric sentiment analysis within the political news domain and highlight the importance of suitable prompting strategies and model architectures.
Paper Structure (22 sections, 5 figures, 4 tables)

This paper contains 22 sections, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Entity Centric Sentiment from Political News
  3. Our Approach
  4. Zero-shot approach
  5. standard zero-shot
  6. 2-stage Prompting
  7. Few-shot approach
  8. Standard few-shot
  9. COT prompting
  10. Self-Consistency
  11. Experimental Setup
  12. Datasets
  13. PerSenT:
  14. Event Registry Data:
  15. Language Models
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Standard Zero-shot Prompting Demonstration. Input prompt includes news article excerpt and target entity phrase, generating entity-specific sentiment output.
  • Figure 2: Two-Stage Prompting for Sentiment Prediction in Zero-Shot Setting. The first stage involves extracting rationales for entity-specific sentiment prediction, providing opinions regarding the target entity. In the second stage, sentiment tags are predicted based on the explanations
  • Figure 3: Standard Few-shot Prompting Illustration. Input: Entity context (news article). Output: Entity-specific sentiment tag (positive/negative/neutral).
  • Figure 4: COT Prompting Demonstration. Input: Entity context (news article), Rationale: Justification of sentiment tag, Output: Entity-specific sentiment (positive/negative/neutral).
  • Figure 5: Chain-of-Thought Prompting with Self-Consistency in few-shot settings. The image illustrates the process where the LLM is provided with demonstration exemplars consisting of (input, output, and rationale) triplets. During inference, given a query input, the LLM returns multiple outputs containing sentiment tags and related explanations. The final sentiment tag is identified through sampling and marginalization technique.