Fine-Tuning Gemma-7B for Enhanced Sentiment Analysis of Financial News Headlines

TL;DR

This study tackles sentiment analysis of financial news headlines from a retail investor viewpoint using fine-tuned large language models. By evaluating distilbert-base-uncased, Llama, and Gemma-7B on the FinancialPhraseBank dataset, the authors show that the fine-tuned Gemma-7B model achieves the best overall performance (accuracy $0.874$) and strong per-class metrics, demonstrating robustness to financial linguistics. The methodology combines PEFT-based fine-tuning with a structured preprocessing and feature-extraction pipeline, yielding a practical tool for market insights, risk management, and investment support. The work highlights the potential of advanced LLMs to transform financial sentiment analysis and suggests avenues for broader datasets and multi-source contextual integration to further enhance performance.

Abstract

In this study, we explore the application of sentiment analysis on financial news headlines to understand investor sentiment. By leveraging Natural Language Processing (NLP) and Large Language Models (LLM), we analyze sentiment from the perspective of retail investors. The FinancialPhraseBank dataset, which contains categorized sentiments of financial news headlines, serves as the basis for our analysis. We fine-tuned several models, including distilbert-base-uncased, Llama, and gemma-7b, to evaluate their effectiveness in sentiment classification. Our experiments demonstrate that the fine-tuned gemma-7b model outperforms others, achieving the highest precision, recall, and F1 score. Specifically, the gemma-7b model showed significant improvements in accuracy after fine-tuning, indicating its robustness in capturing the nuances of financial sentiment. This model can be instrumental in providing market insights, risk management, and aiding investment decisions by accurately predicting the sentiment of financial news. The results highlight the potential of advanced LLMs in transforming how we analyze and interpret financial information, offering a powerful tool for stakeholders in the financial industry.

Figures (6)

• Figure 1: Sentiment Distribution: Box Plot Analysis
• Figure 2: Sentiment Distribution: Donut Chart Analysis
• Figure 3: Feature Correlation Matrix
• Figure 4: Keyword Frequency Distribution
• Figure 5: Confusion Matrix of Model Predictions