Exploring Sentiment Manipulation by LLM-Enabled Intelligent Trading Agents
David Byrd
TL;DR
This paper studies whether an autonomous trading agent powered by a large language model can manipulate financial market prices by generating sentiment-laden social posts observed by a sentiment-aware agent. It builds a TD3-based trading agent within a high-fidelity ABIDES-based market simulator, employing Llama 3.2 for posting and RoBERTa for sentiment analysis, all locally. Results show the agent learns to improve returns in-sample, with mixed out-of-sample results, and that enabling direct manipulation of the sentiment feed yields substantial gains relative to indirect interaction. The authors discuss safety, regulatory implications, and future work, including normative reinforcement learning to govern output and ensure safe deployment of language-controlled autonomous agents in finance.
Abstract
Companies across all economic sectors continue to deploy large language models at a rapid pace. Reinforcement learning is experiencing a resurgence of interest due to its association with the fine-tuning of language models from human feedback. Tool-chain language models control task-specific agents; if the converse has not already appeared, it soon will. In this paper, we present what we believe is the first investigation of an intelligent trading agent based on continuous deep reinforcement learning that also controls a large language model with which it can post to a social media feed observed by other traders. We empirically investigate the performance and impact of such an agent in a simulated financial market, finding that it learns to optimize its total reward, and thereby augment its profit, by manipulating the sentiment of the posts it produces. The paper concludes with discussion, limitations, and suggestions for future work.