Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reinforcement Learning for Dividend Optimization in Partially Observed Regime-Switching Diffusion Model

Zhongqin Gao, Yan Lv, Jingmin He

TL;DR

The paper tackles optimal dividend distribution under partial information in a regime-switching diffusion model with unknown environment parameters. It blends belief-state filtering with an entropy-regularized continuous-time RL framework, deriving semi-analytical structural results that yield a Gibbs-type, state-dependent dividend policy and a two-ODE representation of the value function. An actor-critic learning algorithm (PO-RSEOD) is developed to learn the value function and policy online while filtering the hidden regime; it includes two learning modes (ML minimization and episodic CTD) and robust policy-improvement guarantees within the Gibbs class. Numerical experiments demonstrate strong out-of-sample performance and data-driven adaptability, with careful comparison to FD benchmarks and two environment-estimation methods, highlighting the approach’s practical viability for robust dividend optimization under uncertainty and regime-switching. Overall, the work advances continuous-time RL for partially observed financial control problems by exploiting problem structure to enable efficient learning and reliable policy performance.

Abstract

This paper studies the optimal dividend problem with a bounded payout rate in a partially observed regime-switching diffusion model, where, in practice, the market regime is unobserved and key model parameters are unknown. To address this partial-information setting, we propose a continuous-time reinforcement learning (RL) approach within an exploratory (entropy-regularized) stochastic control framework for discounted dividends under regime switching. The associated exploratory Hamilton-Jacobi-Bellman (HJB) system admits semi-analytical characterizations of the value function and the optimal exploratory dividend policy, determined by two unknown functions solving two ordinary differential equations (ODEs) together with positive real roots of the induced quadratic equations. Exploiting this structure, we introduce parametric families for both the value function and the policy, using low-degree polynomial approximations to the ODE solutions. We then develop an actor-critic RL algorithm to learn the optimal exploratory policy through interactions with the market environment: it performs belief-state filtering from observed data and iterates policy evaluation and policy improvement online to refine the policy. Numerical experiments demonstrate strong out-of-sample performance of the learned dividend policies.

Reinforcement Learning for Dividend Optimization in Partially Observed Regime-Switching Diffusion Model

TL;DR

The paper tackles optimal dividend distribution under partial information in a regime-switching diffusion model with unknown environment parameters. It blends belief-state filtering with an entropy-regularized continuous-time RL framework, deriving semi-analytical structural results that yield a Gibbs-type, state-dependent dividend policy and a two-ODE representation of the value function. An actor-critic learning algorithm (PO-RSEOD) is developed to learn the value function and policy online while filtering the hidden regime; it includes two learning modes (ML minimization and episodic CTD) and robust policy-improvement guarantees within the Gibbs class. Numerical experiments demonstrate strong out-of-sample performance and data-driven adaptability, with careful comparison to FD benchmarks and two environment-estimation methods, highlighting the approach’s practical viability for robust dividend optimization under uncertainty and regime-switching. Overall, the work advances continuous-time RL for partially observed financial control problems by exploiting problem structure to enable efficient learning and reliable policy performance.

Abstract

This paper studies the optimal dividend problem with a bounded payout rate in a partially observed regime-switching diffusion model, where, in practice, the market regime is unobserved and key model parameters are unknown. To address this partial-information setting, we propose a continuous-time reinforcement learning (RL) approach within an exploratory (entropy-regularized) stochastic control framework for discounted dividends under regime switching. The associated exploratory Hamilton-Jacobi-Bellman (HJB) system admits semi-analytical characterizations of the value function and the optimal exploratory dividend policy, determined by two unknown functions solving two ordinary differential equations (ODEs) together with positive real roots of the induced quadratic equations. Exploiting this structure, we introduce parametric families for both the value function and the policy, using low-degree polynomial approximations to the ODE solutions. We then develop an actor-critic RL algorithm to learn the optimal exploratory policy through interactions with the market environment: it performs belief-state filtering from observed data and iterates policy evaluation and policy improvement online to refine the policy. Numerical experiments demonstrate strong out-of-sample performance of the learned dividend policies.
Paper Structure (34 sections, 15 theorems, 81 equations, 4 figures, 3 tables)

This paper contains 34 sections, 15 theorems, 81 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Regime-switching optimal dividend problem under completely observable
  3. Problem formulation
  4. HJB equation and verification theorem
  5. Value function and optimal dividend policy
  6. Regime-switching optimal dividend problem under partially observable
  7. Filtering and innovation
  8. Exploratory problem formulation
  9. Solve to the regime-switching exploratory optimal dividend problem
  10. Exploratory HJB equation and optimal exploratory dividend policy
  11. Verification theorem
  12. Solution to exploratory HJB equation
  13. RL algorithm design
  14. Policy improvement and policy convergence
  15. PO-RSEOD algorithm
  16. ...and 19 more sections

Key Result

Theorem 2.1

Suppose that $\nu: \mathbb{R}_+ \times E \rightarrow \mathbb{R}_+$ satisfies $\nu(\cdot, i)\in \mathbb{C}^{2}((0, \infty)\backslash \mathcal{P}_i) \cap \mathbb{C}^{1}((0, \infty))$ for a finite subset $\mathcal{P}_i \in \mathbb{R}_+$ and $i\in E$. If $\nu(x, i)$ satisfies the following HJB equation with boundary condition $\nu(0,i)=0$, and $|\nu|$, $|\nu_x|$ are bounded, then $\nu(x,i)$ is the va

Figures (4)

  • Figure 1: Surfaces of $V(x, p)$ from FD for dividend caps $\mathbf{a}\in\{0.6, 1, 3\}$ and volatilities $\sigma\in\{0.3, 0.8\}$.
  • Figure 2: $g_i(p)$$(i=1,2)$ and $g(p)$ from FD versus their OLS polynomial fits with order $m=2$.
  • Figure 3: Market regime vs. belief states constructed with true and estimated parameters over 10 years.
  • Figure 4: Left: training convergence of $e^{\gamma_i}$, $i=0,\ldots, 4$ for ML Minimization/Episodic Online CTD(0)/ $\mathrm{CTD}^*(0)$; Right: training paths of value function and regularized HJB loss (averaged over every 5 iterations).

Theorems & Definitions (20)

  • Theorem 2.1: Verification theorem
  • Definition 3.1
  • Proposition 3.1
  • Proposition 4.1
  • Theorem 4.1: Verification theorem
  • Proposition 4.2
  • Remark 4.1
  • Proposition 4.3
  • Proposition 4.4
  • Theorem 4.2
  • ...and 10 more