Are LLMs Rational Investors? A Study on Detecting and Reducing the Financial Bias in LLMs

TL;DR

This work introduces the Financial Bias Indicators (FBI) framework to systematically assess and mitigate financial rationality in LLMs. By combining behavioral finance theory with four modular components—Bias Unveiler, Bias Detective, Bias Tracker, and Bias Antidote—the study quantifies Belief Bias and Risk-preference Bias across 23 LLMs, supplemented by the FinCausal causal-knowledge dataset. Four prompt-based debiasing methods are evaluated, revealing that model size and financial training influence bias in nuanced ways, and that careful prompt design can reduce irrationality without sacrificing capability. The findings highlight persistent financial biases in FinLLMs and provide actionable techniques to improve reliability for financial analysis tasks, with implications for safer AI-assisted market insights.

Abstract

Large Language Models (LLMs) are increasingly adopted in financial analysis for interpreting complex market data and trends. However, their use is challenged by intrinsic biases (e.g., risk-preference bias) and a superficial understanding of market intricacies, necessitating a thorough assessment of their financial insight. To address these issues, we introduce Financial Bias Indicators (FBI), a framework with components like Bias Unveiler, Bias Detective, Bias Tracker, and Bias Antidote to identify, detect, analyze, and eliminate irrational biases in LLMs. By combining behavioral finance principles with bias examination, we evaluate 23 leading LLMs and propose a de-biasing method based on financial causal knowledge. Results show varying degrees of financial irrationality among models, influenced by their design and training. Models trained specifically on financial datasets may exhibit more irrationality, and even larger financial language models (FinLLMs) can show more bias than smaller, general models. We utilize four prompt-based methods incorporating causal debiasing, effectively reducing financial biases in these models. This work enhances the understanding of LLMs' bias in financial applications, laying the foundation for developing more reliable and rational financial analysis tools.

Figures (57)

• Figure 1: An example of model irrationality, where the model gives inconsistent expectations for the same event from different subjects, resulting in different emotions, reflect the financial bias towards the company.
• Figure 2: The framework of FBI consists of the Bias Unveiler, Bias Detective, Bias Tracker, and Bias Antidote. The Bias Unveiler defines financial biases in LLMs based on behavioral finance. The Bias Detective constructs relevant data and detects biases in 23 leading LLMs. The Bias Tracker traces biases using System 2 slow thinking analysis and attention mechanism visualization. The Bias Antidote attempts to debias the models using four methods.
• Figure 3: The score variance between models in event news detection. A higher variance indicates a more severe Anchoring Effect.
• Figure 4: The score distribution between FinQwen and Qwen-14B across 24 news events shows that FinQwen is less stable and more aggressive in scoring compared to Qwen-14B, exhibiting stronger Overconfidence.
• Figure 5: Comparison of risk-preference distribution of three models under different prompt methods.