AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Framework

TL;DR

This work formalizes financial analysis as a dual task of stock trend prediction and financial Q&A, and introduces AlphaFin—a multimodal, CoT-enriched dataset pool—together with the Stock-Chain framework. Stock-Chain combines a two-stage stock-prediction pipeline with a retrieval-augmented Q&A module that leverages a continually updated vector database to ground LLM responses in real-time market knowledge. Through extensive experiments on AlphaFin-Test, Stock-Chain achieves state-of-the-art ARR (≈30.8%) and competitive accuracy, while preference studies and case analyses confirm improved practicality and reduced hallucinations. The approach demonstrates that integrating RAG with fine-tuned FinLLMs on targeted financial data substantially enhances both predictive accuracy and the quality of financial guidance in dynamic markets.

Abstract

The task of financial analysis primarily encompasses two key areas: stock trend prediction and the corresponding financial question answering. Currently, machine learning and deep learning algorithms (ML&DL) have been widely applied for stock trend predictions, leading to significant progress. However, these methods fail to provide reasons for predictions, lacking interpretability and reasoning processes. Also, they can not integrate textual information such as financial news or reports. Meanwhile, large language models (LLMs) have remarkable textual understanding and generation ability. But due to the scarcity of financial training datasets and limited integration with real-time knowledge, LLMs still suffer from hallucinations and are unable to keep up with the latest information. To tackle these challenges, we first release AlphaFin datasets, combining traditional research datasets, real-time financial data, and handwritten chain-of-thought (CoT) data. It has a positive impact on training LLMs for completing financial analysis. We then use AlphaFin datasets to benchmark a state-of-the-art method, called Stock-Chain, for effectively tackling the financial analysis task, which integrates retrieval-augmented generation (RAG) techniques. Extensive experiments are conducted to demonstrate the effectiveness of our framework on financial analysis.

Figures (7)

• Figure 1: An example of the financial analysis task, including stock trend prediction and financial Q&A. Traditional ML&DL methods merely provide uncertain forecasts (Up/Down) without any justification, while original LLMs could offer analysis of the prediction but unhelpful.
• Figure 2: The data source and preprocessing of the proposed AlphaFin datasets.
• Figure 3: An illustration of the Stock-Chain framework of the two stages in financial analysis.
• Figure 4: Accumulated returns (AR) of each baseline under the test set of the financial report dataset from January 2020 to July 2023. The figure shows the curves of some baselines.
• Figure 5: Preference evaluations via human.