FinRLlama: A Solution to LLM-Engineered Signals Challenge at FinRL Contest 2024
Arnav Grover
TL;DR
The paper addresses the challenge of applying large language models to financial sentiment analysis by incorporating market-specific knowledge and temporal dynamics. It introduces a prompt-based framework guided by Reinforcement Learning from Market Feedback to fine-tune LLaMA-3.2-3B-Instruct, integrating historical market data and a reward-based feedback loop to generate actionable sentiment signals. Experimental results across 2020–2023 show FinRLlama delivers more consistent sentiment signals and tighter trading outcomes than a baseline, earning Task II in the FinRL 2024 competition, with code available on GitHub. This work demonstrates a market-aligned approach to adapting LLMs for finance, offering a reproducible methodology for prompt design and RL-based fine-tuning in sentiment-driven trading contexts.
Abstract
In response to Task II of the FinRL Challenge at ACM ICAIF 2024, this study proposes a novel prompt framework for fine-tuning large language models (LLM) with Reinforcement Learning from Market Feedback (RLMF). Our framework incorporates market-specific features and short-term price dynamics to generate more precise trading signals. Traditional LLMs, while competent in sentiment analysis, lack contextual alignment for financial market applications. To bridge this gap, we fine-tune the LLaMA-3.2-3B-Instruct model using a custom RLMF prompt design that integrates historical market data and reward-based feedback. Our evaluation shows that this RLMF-tuned framework outperforms baseline methods in signal consistency and achieving tighter trading outcomes; awarded as winner of Task II. You can find the code for this project on GitHub.