Exposing Product Bias in LLM Investment Recommendation

TL;DR

This work identifies a novel product bias in LLM-based investment recommendations, showing that seven state-of-the-art models consistently favor specific assets (e.g., AAPL, MSFT) and sectors, potentially skewing investor decisions and market dynamics. It introduces a large-scale, automated pipeline generating $567000$ samples across $5$ asset classes to quantify bias with the Gini Index, revealing strong biases especially in stock recommendations. Despite debiasing prompts, biases persist, highlighting limitations of current mitigation approaches and the need for fairness-focused design and transparency. The findings have practical implications for investors and regulators, underscoring the risk of capital concentration and market instability when high-exposure assets are repeatedly promoted by RecLLMs.

Abstract

Large language models (LLMs), as a new generation of recommendation engines, possess powerful summarization and data analysis capabilities, surpassing traditional recommendation systems in both scope and performance. One promising application is investment recommendation. In this paper, we reveal a novel product bias in LLM investment recommendation, where LLMs exhibit systematic preferences for specific products. Such preferences can subtly influence user investment decisions, potentially leading to inflated valuations of products and financial bubbles, posing risks to both individual investors and market stability. To comprehensively study the product bias, we develop an automated pipeline to create a dataset of 567,000 samples across five asset classes (stocks, mutual funds, cryptocurrencies, savings, and portfolios). With this dataset, we present the bf first study on product bias in LLM investment recommendations. Our findings reveal that LLMs exhibit clear product preferences, such as certain stocks (e.g., `AAPL' from Apple and `MSFT' from Microsoft). Notably, this bias persists even after applying debiasing techniques. We urge AI researchers to take heed of the product bias in LLM investment recommendations and its implications, ensuring fairness and security in the digital space and market.

Figures (7)

• Figure 1: Distribution of preferred products in stock investment.
• Figure 2: Distribution of preferred asset classes in portfolio (Investment Amount).
• Figure 3: Recommendation overlap across different LLMs
• Figure 4: Distribution of preferred asset classes in portfolio (Recommendation Frequency).
• Figure 5: Distribution of preferred products in mutual fund investment.