Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Designing Heterogeneous LLM Agents for Financial Sentiment Analysis

Frank Xing

TL;DR

This paper introduces Heterogeneous Agent Discussion (HAD), a kernel-theory–based design framework that uses specialized, non-finetuned LLM agents to perform financial sentiment analysis through aggregated discussions. HAD targets common FSA error types with mood, rhetoric, dependency, aspect, and reference agents, and demonstrates consistent improvements across multiple FSA datasets, particularly when agent discussions are rich. The study provides empirical, ablation, and case-study evidence that error-type-guided heterogeneity among LLM agents can close a portion of the gap between prompting and fine-tuning, offering a practical, theory-informed path for LLM-based FSA systems. It also discusses limitations such as computational cost, scalability, and data confidentiality, outlining future directions for refining agent designs and expanding applications in financial decision-making.

Abstract

Large language models (LLMs) have drastically changed the possible ways to design intelligent systems, shifting the focuses from massive data acquisition and new modeling training to human alignment and strategical elicitation of the full potential of existing pre-trained models. This paradigm shift, however, is not fully realized in financial sentiment analysis (FSA), due to the discriminative nature of this task and a lack of prescriptive knowledge of how to leverage generative models in such a context. This study investigates the effectiveness of the new paradigm, i.e., using LLMs without fine-tuning for FSA. Rooted in Minsky's theory of mind and emotions, a design framework with heterogeneous LLM agents is proposed. The framework instantiates specialized agents using prior domain knowledge of the types of FSA errors and reasons on the aggregated agent discussions. Comprehensive evaluation on FSA datasets show that the framework yields better accuracies, especially when the discussions are substantial. This study contributes to the design foundations and paves new avenues for LLMs-based FSA. Implications on business and management are also discussed.

Designing Heterogeneous LLM Agents for Financial Sentiment Analysis

TL;DR

This paper introduces Heterogeneous Agent Discussion (HAD), a kernel-theory–based design framework that uses specialized, non-finetuned LLM agents to perform financial sentiment analysis through aggregated discussions. HAD targets common FSA error types with mood, rhetoric, dependency, aspect, and reference agents, and demonstrates consistent improvements across multiple FSA datasets, particularly when agent discussions are rich. The study provides empirical, ablation, and case-study evidence that error-type-guided heterogeneity among LLM agents can close a portion of the gap between prompting and fine-tuning, offering a practical, theory-informed path for LLM-based FSA systems. It also discusses limitations such as computational cost, scalability, and data confidentiality, outlining future directions for refining agent designs and expanding applications in financial decision-making.

Abstract

Large language models (LLMs) have drastically changed the possible ways to design intelligent systems, shifting the focuses from massive data acquisition and new modeling training to human alignment and strategical elicitation of the full potential of existing pre-trained models. This paradigm shift, however, is not fully realized in financial sentiment analysis (FSA), due to the discriminative nature of this task and a lack of prescriptive knowledge of how to leverage generative models in such a context. This study investigates the effectiveness of the new paradigm, i.e., using LLMs without fine-tuning for FSA. Rooted in Minsky's theory of mind and emotions, a design framework with heterogeneous LLM agents is proposed. The framework instantiates specialized agents using prior domain knowledge of the types of FSA errors and reasons on the aggregated agent discussions. Comprehensive evaluation on FSA datasets show that the framework yields better accuracies, especially when the discussions are substantial. This study contributes to the design foundations and paves new avenues for LLMs-based FSA. Implications on business and management are also discussed.
Paper Structure (12 sections, 3 figures, 4 tables)

This paper contains 12 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work and Design Process
  3. Using LLMs for Financial Sentiment Analysis
  4. Prompt Engineering
  5. Kernel Theory: Emotions and the Society of Mind
  6. Meta-requirements, Meta-designs, and Hypotheses
  7. Design Artifact: Heterogeneous Agent Discussion (HAD)
  8. Evaluation
  9. Performance Improvement
  10. Ablation Analysis
  11. Case Study
  12. Discussion, Conclusion, and Future Work

Figures (3)

  • Figure 1: Different multi-agent LLM frameworks for reaching a consensus: (a) homogenous multi-agent debate md, (b) multi-role multi-agent negotiation mn, (c) heterogeneous multi-agent discussion (HAD: the proposed framework). Colors denote different roles and shapes denote heterogeneous agents.
  • Figure 2: Illustration of the generation of emotional states from activating a collection of resources, cf. pg. 4 in emotion-machine.
  • Figure 3: An illustrative comparison between naive prompting (the upper example) and the proposed HAD framework (the lower example) with 3 heterogeneous agents inspired by FSA error types.