Multimodal Gen-AI for Fundamental Investment Research

TL;DR

The paper addresses automating fundamental investment research by fine-tuning large language models to synthesize information and generate investment ideas. It systematically compares unsupervised LoRA fine-tuning of Llama2, supervised text–price fine-tuning, and instruction fine-tuning of GPT-3.5 on domain-specific corpora, including institutional reports and market data. Through perplexity, generalizability benchmarks, and human evaluations, the study shows that domain-focused fine-tuning improves text modeling, summarization, reasoning, and finance-domain question answering, with instruction-tuned GPT-3.5 delivering particularly analyst-like, relevant outputs. The work demonstrates a practical pathway toward an AI agent that can support or augment human investors in routine analytical tasks, with broader implications for inventory of institutional research data and time-series-informed decision making.

Abstract

This report outlines a transformative initiative in the financial investment industry, where the conventional decision-making process, laden with labor-intensive tasks such as sifting through voluminous documents, is being reimagined. Leveraging language models, our experiments aim to automate information summarization and investment idea generation. We seek to evaluate the effectiveness of fine-tuning methods on a base model (Llama2) to achieve specific application-level goals, including providing insights into the impact of events on companies and sectors, understanding market condition relationships, generating investor-aligned investment ideas, and formatting results with stock recommendations and detailed explanations. Through state-of-the-art generative modeling techniques, the ultimate objective is to develop an AI agent prototype, liberating human investors from repetitive tasks and allowing a focus on high-level strategic thinking. The project encompasses a diverse corpus dataset, including research reports, investment memos, market news, and extensive time-series market data. We conducted three experiments applying unsupervised and supervised LoRA fine-tuning on the llama2_7b_hf_chat as the base model, as well as instruction fine-tuning on the GPT3.5 model. Statistical and human evaluations both show that the fine-tuned versions perform better in solving text modeling, summarization, reasoning, and finance domain questions, demonstrating a pivotal step towards enhancing decision-making processes in the financial domain. Code implementation for the project can be found on GitHub: https://github.com/Firenze11/finance_lm.

Figures (9)

• Figure 1: Detailed settings of LoRA.
• Figure 2: Unsupervised fine-tuning of Llama2 using LoRA
• Figure 3: Supervised fine-tuning of Llama2 using LoRA, where the text comes from news headlines and targets are labels indicating stock price changes.
• Figure 4: Instruction fine-tuning of GPT3.5 using question and answer pairs intended to impart investment principles.
• Figure 5: Comparison of eval set loss for the baseline and fine-tuned Llama2 models.