Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fine-tuning Timeseries Predictors Using Reinforcement Learning

Hugo Cazaux, Ralph Rudd, Hlynur Stefánsson, Sverrir Ólafsson, Eyjólfur Ingi Ásgeirsson

Abstract

This chapter presents three major reinforcement learning algorithms used for fine-tuning financial forecasters. We propose a clear implementation plan for backpropagating the loss of a reinforcement learning task to a model trained using supervised learning, and compare the performance before and after the fine-tuning. We find an increase in performance after fine-tuning, and transfer learning properties to the models, indicating the benefits of fine-tuning. We also highlight the tuning process and empirical results for future implementation by practitioners.

Fine-tuning Timeseries Predictors Using Reinforcement Learning

Abstract

This chapter presents three major reinforcement learning algorithms used for fine-tuning financial forecasters. We propose a clear implementation plan for backpropagating the loss of a reinforcement learning task to a model trained using supervised learning, and compare the performance before and after the fine-tuning. We find an increase in performance after fine-tuning, and transfer learning properties to the models, indicating the benefits of fine-tuning. We also highlight the tuning process and empirical results for future implementation by practitioners.
Paper Structure (20 sections, 1 figure, 9 tables)

This paper contains 20 sections, 1 figure, 9 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Data
  4. Financial and ESG Data
  5. MuJoCo Benchmarking Environments
  6. Framework Details
  7. Proximal Policy Optimization (PPO)
  8. Centralized Multi-Agent PPO (CMAPPO)
  9. Group Relative Policy Optimization (GRPO)
  10. Design of the Reinforcement Learning Environment
  11. Latent Representation versus Actor Network
  12. Benchmarking
  13. Results
  14. Fine-tuning and Frozen Layers
  15. Transfer Learning
  16. ...and 5 more sections

Figures (1)

  • Figure 1: Training time vs MSE. Dotted line is the original model performance before fine-tuning. Training time is scaled down from 1e6 for readability.