Enhancing Emotion Prediction in News Headlines: Insights from ChatGPT and Seq2Seq Models for Free-Text Generation

TL;DR

This work tackles predicting readers' emotions elicited by news headlines by leveraging free-text emotion explanations rather than relying on headlines alone. It compares sequence-to-sequence generation ( Headlines2Explanations ) and large-language-model–driven approaches (ChatGPT/GPT-4) to produce emotion explanations from headlines, which are then used for emotion classification, including intermediate-task transfer learning with T5. The results show that incorporating free-text explanations substantially improves exact and top-2 emotion prediction accuracy, with GPT-generated explanations achieving strong performance and interpretability; few-shot prompting further enhances alignment with human emotion distributions. The findings suggest practical applications for newsroom analytics and audience engagement tools, while highlighting limitations such as single ground-truth labeling and potential LLM biases, underscoring the need for context-aware, distribution-focused emotion prediction.

Abstract

Predicting emotions elicited by news headlines can be challenging as the task is largely influenced by the varying nature of people's interpretations and backgrounds. Previous works have explored classifying discrete emotions directly from news headlines. We provide a different approach to tackling this problem by utilizing people's explanations of their emotion, written in free-text, on how they feel after reading a news headline. Using the dataset BU-NEmo+ (Gao et al., 2022), we found that for emotion classification, the free-text explanations have a strong correlation with the dominant emotion elicited by the headlines. The free-text explanations also contain more sentimental context than the news headlines alone and can serve as a better input to emotion classification models. Therefore, in this work we explored generating emotion explanations from headlines by training a sequence-to-sequence transformer model and by using pretrained large language model, ChatGPT (GPT-4). We then used the generated emotion explanations for emotion classification. In addition, we also experimented with training the pretrained T5 model for the intermediate task of explanation generation before fine-tuning it for emotion classification. Using McNemar's significance test, methods that incorporate GPT-generated free-text emotion explanations demonstrated significant improvement (P-value < 0.05) in emotion classification from headlines, compared to methods that only use headlines. This underscores the value of using intermediate free-text explanations for emotion prediction tasks with headlines.

Figures (8)

• Figure 1: The interface for the gun violence emotion annotation crowdsourcing experiment for a single news headline.
• Figure 2: Examples of headlines that do not contain the sentiment information of the reactions.
• Figure 3: CEE-T. Sequence-to-sequence emotion classification architecture.
• Figure 4: T5 with transfer learning. Pretrained T5 is first fine-tuned on headline to concatenated emotion explanation pairs and subsequently on headline to emotion pairs.
• Figure 5: Text prompt and generation format of the zero-shot free-text emotion explanation generation with ChatGPT. In the generation, each free-text explanation is followed with an emotion label after the semicolon.