Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Once Burned, Twice Shy? The Effect of Stock Market Bubbles on Traders that Learn by Experience

Haibei Zhu, Svitlana Vyetrenko, Serafin Grundl, David Byrd, Kshama Dwarakanath, Tucker Balch

TL;DR

This paper investigates how experience with stock market bubbles shapes reinforcement learning traders in a multi-agent market environment (ABIDES). By training agents with varying levels of bubble exposure and evaluating them in bubble-rich scenarios, the authors show that bubble experience pushes agents from momentum-based to value-based strategies, which in turn suppress bubble formation, magnitude, and duration. The approach combines PPO-trained RL agents, SHAP feature attribution, and controlled bubble-generation mechanics (including a mean-reverting fundamental model) to quantify the impact of experience on market dynamics. The findings imply that learning from rare, extreme market events can modulate boom-bust cycles, with memory effects that gradually fade yet offer potential stability benefits for autonomous trading systems.

Abstract

We study how experience with asset price bubbles changes the trading strategies of reinforcement learning (RL) traders and ask whether the change in trading strategies helps to prevent future bubbles. We train the RL traders in a multi-agent market simulation platform, ABIDES, and compare the strategies of traders trained with and without bubble experience. We find that RL traders without bubble experience behave like short-term momentum traders, whereas traders with bubble experience behave like value traders. Therefore, RL traders without bubble experience amplify bubbles, whereas RL traders with bubble experience tend to suppress and sometimes prevent them. This finding suggests that learning from experience is a mechanism for a boom and bust cycle where the experience of a collapsing bubble makes future bubbles less likely for a period of time until the memory fades and bubbles become more likely to form again.

Once Burned, Twice Shy? The Effect of Stock Market Bubbles on Traders that Learn by Experience

TL;DR

This paper investigates how experience with stock market bubbles shapes reinforcement learning traders in a multi-agent market environment (ABIDES). By training agents with varying levels of bubble exposure and evaluating them in bubble-rich scenarios, the authors show that bubble experience pushes agents from momentum-based to value-based strategies, which in turn suppress bubble formation, magnitude, and duration. The approach combines PPO-trained RL agents, SHAP feature attribution, and controlled bubble-generation mechanics (including a mean-reverting fundamental model) to quantify the impact of experience on market dynamics. The findings imply that learning from rare, extreme market events can modulate boom-bust cycles, with memory effects that gradually fade yet offer potential stability benefits for autonomous trading systems.

Abstract

We study how experience with asset price bubbles changes the trading strategies of reinforcement learning (RL) traders and ask whether the change in trading strategies helps to prevent future bubbles. We train the RL traders in a multi-agent market simulation platform, ABIDES, and compare the strategies of traders trained with and without bubble experience. We find that RL traders without bubble experience behave like short-term momentum traders, whereas traders with bubble experience behave like value traders. Therefore, RL traders without bubble experience amplify bubbles, whereas RL traders with bubble experience tend to suppress and sometimes prevent them. This finding suggests that learning from experience is a mechanism for a boom and bust cycle where the experience of a collapsing bubble makes future bubbles less likely for a period of time until the memory fades and bubbles become more likely to form again.
Paper Structure (16 sections, 5 figures, 2 tables)

This paper contains 16 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Experiment
  4. Generating and Bursting Bubbles
  5. Bubble Detection
  6. Reinforcement Learning Agent Training and Testing
  7. Results
  8. Profit Analysis
  9. Bubble Measurement Metrics
  10. Bubble Count
  11. Bubble Magnitude
  12. Bubble Duration
  13. Discussion
  14. Bubble Formation
  15. RL Agent Strategy Analyses
  16. ...and 1 more sections

Figures (5)

  • Figure 1: The start and end of a bubble are identified by comparison of moving averages on the market price with the "fundamental" value of the asset.
  • Figure 2: Profit analysis and bubble count testing results.
  • Figure 3: Bubble magnitude and duration testing results.
  • Figure 4: A bubble scenario example with RL agent holding position and marked-to-market profit.
  • Figure 5: Cumulative SHAP analysis for both agents in the example bubble scenario.