Generative AI for Stock Selection

TL;DR

This work investigates whether generative AI, guided by retrieval-augmented generation and programmatic prompting, can automate the discovery of actionable, structured features for cross-sectional stock selection. The authors compare Baseline, Structured Prompting, and DSPy programmatic prompting configurations across three rich data sources (analyst forecasts, options, and price-volume), using a gradient-boosted model with point-in-time constraints and a daily long-short portfolio framework. Across multiple out-of-sample windows, AI-generated features yield systematic, orthogonal alpha and improved risk-adjusted performance, with DSPy achieving the strongest net Sharpe under realistic costs, while retrieval quality critically governs outcomes. The findings suggest LLM-assisted feature discovery can meaningfully augment traditional quantitative pipelines, provided that domain knowledge is well-curated, prompts are governance-aware, and backtesting hygiene is strictly maintained. Practically, the study highlights the importance of knowledge-base quality, interpretable AI-generated transformations, and the balance between automated prompting and human-guided engineering in delivering robust, deployable signals for asset management.

Abstract

We study whether generative AI can automate feature discovery in U.S. equities. Using large language models with retrieval-augmented generation and structured/programmatic prompting, we synthesize economically motivated features from analyst, options, and price-volume data. These features are then used as inputs to a tabular machine-learning model to forecast short-horizon returns. Across multiple datasets, AI-generated features are consistently competitive with baselines, with Sharpe improvements ranging from 14% to 91% depending on dataset and configuration. Retrieval quality is pivotal: better knowledge bases materially improve outcomes. The AI-generated signals are weakly correlated with traditional features, supporting combination. Overall, generative AI can meaningfully augment feature discovery when retrieval quality is controlled, producing interpretable signals while reducing manual engineering effort.