Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Can LLM-based Financial Investing Strategies Outperform the Market in Long Run?

Weixian Waylon Li, Hyeonjun Kim, Mihai Cucuringu, Tiejun Ma

TL;DR

This paper questions whether LLM-based investing strategies can consistently outperform the market when evaluated under robust, long-horizon conditions. It introduces FINSABER, a bias-aware backtesting framework that uses 20 years of multi-source data and a broad symbol universe to rigorously assess timing- and selection-based strategies, explicitly mitigating survivorship, look-ahead, and data-snooping biases. Across extended backtests and regime analyses, the study finds that previously reported LLM advantages largely disappear, with LLM strategies underperforming passive benchmarks in bull markets and suffering losses in bear markets due to poor risk controls. The work argues that improvements should focus on regime-aware decision-making and adaptive risk management rather than increasing model complexity, and it highlights the substantial costs of LLM backtesting, advocating cost-efficient benchmarking approaches for future research.

Abstract

Large Language Models (LLMs) have recently been leveraged for asset pricing tasks and stock trading applications, enabling AI agents to generate investment decisions from unstructured financial data. However, most evaluations of LLM timing-based investing strategies are conducted on narrow timeframes and limited stock universes, overstating effectiveness due to survivorship and data-snooping biases. We critically assess their generalizability and robustness by proposing FINSABER, a backtesting framework evaluating timing-based strategies across longer periods and a larger universe of symbols. Systematic backtests over two decades and 100+ symbols reveal that previously reported LLM advantages deteriorate significantly under broader cross-section and over a longer-term evaluation. Our market regime analysis further demonstrates that LLM strategies are overly conservative in bull markets, underperforming passive benchmarks, and overly aggressive in bear markets, incurring heavy losses. These findings highlight the need to develop LLM strategies that are able to prioritise trend detection and regime-aware risk controls over mere scaling of framework complexity.

Can LLM-based Financial Investing Strategies Outperform the Market in Long Run?

TL;DR

This paper questions whether LLM-based investing strategies can consistently outperform the market when evaluated under robust, long-horizon conditions. It introduces FINSABER, a bias-aware backtesting framework that uses 20 years of multi-source data and a broad symbol universe to rigorously assess timing- and selection-based strategies, explicitly mitigating survivorship, look-ahead, and data-snooping biases. Across extended backtests and regime analyses, the study finds that previously reported LLM advantages largely disappear, with LLM strategies underperforming passive benchmarks in bull markets and suffering losses in bear markets due to poor risk controls. The work argues that improvements should focus on regime-aware decision-making and adaptive risk management rather than increasing model complexity, and it highlights the substantial costs of LLM backtesting, advocating cost-efficient benchmarking approaches for future research.

Abstract

Large Language Models (LLMs) have recently been leveraged for asset pricing tasks and stock trading applications, enabling AI agents to generate investment decisions from unstructured financial data. However, most evaluations of LLM timing-based investing strategies are conducted on narrow timeframes and limited stock universes, overstating effectiveness due to survivorship and data-snooping biases. We critically assess their generalizability and robustness by proposing FINSABER, a backtesting framework evaluating timing-based strategies across longer periods and a larger universe of symbols. Systematic backtests over two decades and 100+ symbols reveal that previously reported LLM advantages deteriorate significantly under broader cross-section and over a longer-term evaluation. Our market regime analysis further demonstrates that LLM strategies are overly conservative in bull markets, underperforming passive benchmarks, and overly aggressive in bear markets, incurring heavy losses. These findings highlight the need to develop LLM strategies that are able to prioritise trend detection and regime-aware risk controls over mere scaling of framework complexity.
Paper Structure (61 sections, 8 equations, 3 figures, 8 tables)

This paper contains 61 sections, 8 equations, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Definitions of Investing Strategies
  4. Timing-Based Strategies
  5. Selection-Based Strategies
  6. Biases and Robustness Challenges in Backtesting LLM Investors
  7. Survivorship Bias.
  8. Look-ahead Bias.
  9. Data-snooping Bias.
  10. Bias-Mitigation Requires Broader and Longer Evaluation
  11. FINSABER
  12. Multi-source Data for LLM Benchmarking.
  13. Strategies Base.
  14. Two-Step Pipeline for Bias Mitigation.
  15. Evaluation Metrics.
  16. ...and 46 more sections

Figures (3)

  • Figure 1: Overview of the FINSABER Backtest Framework. The central pipeline illustrates the backtesting process. The framework includes a Strategies Base Module (green), which covers both selection-based and timing-based strategies, and a Multi-source Data Module (yellow), integrating diverse financial data inputs.
  • Figure 2: Average Sharpe ratio by regime for all benchmarking strategies. Green = strong, red = weak.
  • Figure 3: Comparative underwater plots for the FinMem (blue) and FinAgent (red) strategies against the Buy and Hold (SPX) benchmark across individual stocks selected in the Composite setup. The plots are grouped by the market regime of the period shown: bull markets (top two rows), bear market (third row), and sideways markets (bottom two rows).